arXiv subjects

Siting Liu

Publications and source records attributed to Siting Liu.

At least 19 recordsLinked to original sources

arXiv2026

Truncated Differentiation Through Primal-Dual Solvers for Inverse Potential Mean-Field Games

We study inverse potential mean-field games (MFGs), in which an unknown spatial inverse-cost (mobility) map is inferred from observed population densities. We solve the forward MFG with a preconditioned primal-dual hybrid gradient (PDHG) method and develop Jacobian-free backpropagation (JFB-$r$), which records only the final $r$ iterations from a detached warm start while retaining the full forward solve. To analyze this truncated differentiation method, we show that the exact-proximal dual-extrapolated PDHG map is a metric resolvent of the maximal monotone KKT operator. This resolvent view shows that JFB-$r$ exactly differentiates a finite-trajectory surrogate and, under a locally fixed active set at an exact equilibrium detach point, converges to the implicit gradient as the tracked depth increases. Across several inverse-MFG settings, numerical experiments show that JFB-r at moderate tracked depths can achieve recovery accuracy comparable to full unrolling while reducing memory and runtime.

math.OC↗

arXiv2025

Recent Advances in Numerical Solutions for Hamilton-Jacobi PDEs

Hamilton-Jacobi partial differential equations (HJ PDEs) play a central role in many applications such as economics, physics, and engineering. These equations describe the evolution of a value function which encodes valuable information about the system, such as action, cost, or level sets of a dynamic process. Their importance lies in their ability to model diverse phenomena, ranging from the propagation of fronts in computational physics to optimal decision-making in control systems. This paper provides a review of some recent advances in numerical methods to address challenges such as high-dimensionality, nonlinearity, and computational efficiency. By examining these developments, this paper sheds light on important techniques and emerging directions in the numerical solution of HJ PDEs.

math.NA↗

arXiv2025

Probabilistic operator learning: generative modeling and uncertainty quantification for foundation models of differential equations

In-context operator networks (ICON) are a class of operator learning methods based on the novel architectures of foundation models. Trained on a diverse set of datasets of initial and boundary conditions paired with corresponding solutions to ordinary and partial differential equations (ODEs and PDEs), ICON learns to map example condition-solution pairs of a given differential equation to an approximation of its solution operator. Here, we present a probabilistic framework that reveals ICON as implicitly performing Bayesian inference, where it computes the mean of the posterior predictive distribution over solution operators conditioned on the provided context, i.e., example condition-solution pairs. The formalism of random differential equations provides the probabilistic framework for describing the tasks ICON accomplishes while also providing a basis for understanding other multi-operator learning methods. This probabilistic perspective provides a basis for extending ICON to \emph{generative} settings, where one can sample from the posterior predictive distribution of solution operators. The generative formulation of ICON (GenICON) captures the underlying uncertainty in the solution operator, which enables principled uncertainty quantification in the solution predictions in operator learning.

stat.ML↗

arXiv2025

iTRI-QA: a Toolset for Customized Question-Answer Dataset Generation Using Language Models for Enhanced Scientific Research

The exponential growth of AI in science necessitates efficient and scalable solutions for retrieving and preserving research information. Here, we present a tool for the development of a customized question-answer (QA) dataset, called Interactive Trained Research Innovator (iTRI) - QA, tailored for the needs of researchers leveraging language models (LMs) to retrieve scientific knowledge in a QA format. Our approach integrates curated QA datasets with a specialized research paper dataset to enhance responses' contextual relevance and accuracy using fine-tuned LM. The framework comprises four key steps: (1) the generation of high-quality and human-generated QA examples, (2) the creation of a structured research paper database, (3) the fine-tuning of LMs using domain-specific QA examples, and (4) the generation of QA dataset that align with user queries and the curated database. This pipeline provides a dynamic and domain-specific QA system that augments the utility of LMs in academic research that will be applied for future research LM deployment. We demonstrate the feasibility and scalability of our tool for streamlining knowledge retrieval in scientific contexts, paving the way for its integration into broader multi-disciplinary applications.

cs.IR↗

arXiv2024

Analytical Heterogeneous Die-to-Die 3D Placement with Macros

This paper presents an innovative approach to 3D mixed-size placement in heterogeneous face-to-face (F2F) bonded 3D ICs. We propose an analytical framework that utilizes a dedicated density model and a bistratal wirelength model, effectively handling macros and standard cells in a 3D solution space. A novel 3D preconditioner is developed to resolve the topological and physical gap between macros and standard cells. Additionally, we propose a mixed-integer linear programming (MILP) formulation for macro rotation to optimize wirelength. Our framework is implemented with full-scale GPU acceleration, leveraging an adaptive 3D density accumulation algorithm and an incremental wirelength gradient algorithm. Experimental results on ICCAD 2023 contest benchmarks demonstrate that our framework can achieve 5.9% quality score improvement compared to the first-place winner with 4.0x runtime speedup. Additional experiments on modern RISC-V designs further validate the generalizability and superiority of our framework.

cs.AR↗

arXiv2024

ApproxPilot: A GNN-based Accelerator Approximation Framework

A typical optimization of customized accelerators for error-tolerant applications such as multimedia, recognition, and classification is to replace traditional arithmetic units like multipliers and adders with the approximate ones to enhance energy efficiency while adhering to accuracy requirements. However, the plethora of arithmetic units and diverse approximate unit options result in an exceedingly large design space. Therefore, there is a pressing need for an end-to-end design framework capable of navigating this intricate design space for approximation optimization. Traditional methods relying on simulation-based or blackbox model evaluations suffer from either high computational costs or limitations in accuracy and scalability, posing significant challenges to the optimization process. In this paper, we propose a Graph Neural Network (GNN) model that leverages the physical connections of arithmetic units to capture their influence on the performance, power, area (PPA), and accuracy of the accelerator. Particularly, we notice that critical path plays a key role in node feature of the GNN model and having it embedded in the feature vector greatly enhances the prediction quality of the models. On top of the models that allow rapid and efficient PPA and accuracy prediction of various approximate accelerator configurations, we can further explore the large design space effectively and build an end-to-end accelerator approximation framework named ApproxPilot to optimize the accelerator approximation. Our experimental results demonstrate that ApproxPilot outperforms state-of-the-art approximation optimization frameworks in both performance and hardware overhead with the same accuracy constraints.

cs.AR↗

arXiv2024

Monotone inclusion methods for a class of second-order non-potential mean-field games

We propose a monotone splitting algorithm for solving a class of second-order non-potential mean-field games. Following [Achdou, Capuzzo-Dolcetta, "Mean Field Games: Numerical Methods," SINUM (2010)], we introduce a finite-difference scheme and observe that the scheme represents first-order optimality conditions for a primal-dual pair of monotone inclusions. Based on this observation, we prove that the finite-difference system obtains a solution that can be provably recovered by an extension of the celebrated primal-dual hybrid gradient (PDHG) algorithm.

math.OC↗

arXiv2024

A Primal-dual hybrid gradient method for solving optimal control problems and the corresponding Hamilton-Jacobi PDEs

Optimal control problems are crucial in various domains, including path planning, robotics, and humanoid control, demonstrating their broad applicability. The connection between optimal control and Hamilton-Jacobi (HJ) partial differential equations (PDEs) underscores the need for solving HJ PDEs to address these control problems effectively. While numerous numerical methods exist for tackling HJ PDEs across different dimensions, this paper introduces an innovative optimization-based approach that reformulates optimal control problems and HJ PDEs into a saddle point problem using a Lagrange multiplier. Our method, based on the preconditioned primal-dual hybrid gradient (PDHG) method, offers a solution to HJ PDEs with first-order accuracy and numerical unconditional stability, enabling larger time steps and avoiding the limitations of explicit time discretization methods. Our approach has ability to handle a wide variety of Hamiltonian functions, including those that are non-smooth and dependent on time and space, through a simplified saddle point formulation that facilitates easy and parallelizable updates. Furthermore, our framework extends to viscous HJ PDEs and stochastic optimal control problems, showcasing its versatility. Through a series of numerical examples, we demonstrate the method's effectiveness in managing diverse Hamiltonians and achieving efficient parallel computation, highlighting its potential for wide-ranging applications in optimal control and beyond.

math.OC↗

arXiv2024

Wasserstein proximal operators describe score-based generative models and resolve memorization

We focus on the fundamental mathematical structure of score-based generative models (SGMs). We first formulate SGMs in terms of the Wasserstein proximal operator (WPO) and demonstrate that, via mean-field games (MFGs), the WPO formulation reveals mathematical structure that describes the inductive bias of diffusion and score-based models. In particular, MFGs yield optimality conditions in the form of a pair of coupled partial differential equations: a forward-controlled Fokker-Planck (FP) equation, and a backward Hamilton-Jacobi-Bellman (HJB) equation. Via a Cole-Hopf transformation and taking advantage of the fact that the cross-entropy can be related to a linear functional of the density, we show that the HJB equation is an uncontrolled FP equation. Second, with the mathematical structure at hand, we present an interpretable kernel-based model for the score function which dramatically improves the performance of SGMs in terms of training samples and training time. In addition, the WPO-informed kernel model is explicitly constructed to avoid the recently studied memorization effects of score-based generative models. The mathematical form of the new kernel-based models in combination with the use of the terminal condition of the MFG reveals new explanations for the manifold learning and generalization properties of SGMs, and provides a resolution to their memorization effects. Finally, our mathematically informed, interpretable kernel-based model suggests new scalable bespoke neural network architectures for high-dimensional applications.

stat.ML↗

arXiv2024

Fine-Tune Language Models as Multi-Modal Differential Equation Solvers

In the growing domain of scientific machine learning, in-context operator learning has shown notable potential in building foundation models, as in this framework the model is trained to learn operators and solve differential equations using prompted data, during the inference stage without weight updates. However, the current model's overdependence on function data overlooks the invaluable human insight into the operator. To address this, we present a transformation of in-context operator learning into a multi-modal paradigm. In particular, we take inspiration from the recent success of large language models, and propose using "captions" to integrate human knowledge about the operator, expressed through natural language descriptions and equations. Also, we introduce a novel approach to train a language-model-like architecture, or directly fine-tune existing language models, for in-context operator learning. We beat the baseline on single-modal learning tasks, and also demonstrated the effectiveness of multi-modal learning in enhancing performance and reducing function data requirements. The proposed method not only significantly enhanced the development of the in-context operator learning paradigm, but also created a new path for the application of language models.

cs.LG↗

arXiv2023

Primal-dual hybrid gradient algorithms for computing time-implicit Hamilton-Jacobi equations

Hamilton-Jacobi (HJ) partial differential equations (PDEs) have diverse applications spanning physics, optimal control, game theory, and imaging sciences. This research introduces a first-order optimization-based technique for HJ PDEs, which formulates the time-implicit update of HJ PDEs as saddle point problems. We remark that the saddle point formulation for HJ equations is aligned with the primal-dual formulation of optimal transport and potential mean-field games (MFGs). This connection enables us to extend MFG techniques and design numerical schemes for solving HJ PDEs. We employ the primal-dual hybrid gradient (PDHG) method to solve the saddle point problems, benefiting from the simple structures that enable fast computations in updates. Remarkably, the method caters to a broader range of Hamiltonians, encompassing non-smooth and spatiotemporally dependent cases. The approach's effectiveness is verified through various numerical examples in both one-dimensional and two-dimensional examples, such as quadratic and $L^1$ Hamiltonians with spatial and time dependence.

math.NA↗

arXiv2023

In-Context Operator Learning with Data Prompts for Differential Equation Problems

This paper introduces a new neural-network-based approach, namely In-Context Operator Networks (ICON), to simultaneously learn operators from the prompted data and apply it to new questions during the inference stage, without any weight update. Existing methods are limited to using a neural network to approximate a specific equation solution or a specific operator, requiring retraining when switching to a new problem with different equations. By training a single neural network as an operator learner, we can not only get rid of retraining (even fine-tuning) the neural network for new problems, but also leverage the commonalities shared across operators so that only a few demos in the prompt are needed when learning a new operator. Our numerical results show the neural network's capability as a few-shot operator learner for a diversified type of differential equation problems, including forward and inverse problems of ordinary differential equations (ODEs), partial differential equations (PDEs), and mean-field control (MFC) problems, and also show that it can generalize its learning capability to operators beyond the training distribution.

cs.LG↗

arXiv2023

A first-order computational algorithm for reaction-diffusion type equations via primal-dual hybrid gradient method

We propose an easy-to-implement iterative method for resolving the implicit (or semi-implicit) schemes arising in solving reaction-diffusion (RD) type equations. We formulate the nonlinear time implicit scheme as a min-max saddle point problem and then apply the primal-dual hybrid gradient (PDHG) method. Suitable precondition matrices are applied to the PDHG method to accelerate the convergence of algorithms under different circumstances. Furthermore, our method is applicable to various discrete numerical schemes with high flexibility. From various numerical examples tested in this paper, the proposed method converges properly and can efficiently produce numerical solutions with sufficient accuracy.

math.NA↗

arXiv2023

Radically Uniform Spike Trains in Optically Injected Quantum Cascade Oscillators

It has been found that noise-induced excitability in quantum well and quantum dot semiconductor laser systems usually produces spike patterns of non-uniform amplitude. In this letter, we experimentally record that an inter-subband quantum cascade laser injected with a monochromatic laser exhibits a series of highly-uniform spike trains in the time domain. Theoretical analysis demonstrates that such high uniformity has its origin in the ultrashort carrier lifetime of the quantum cascade laser gain medium that is typically close to one picosecond.

physics.optics↗

arXiv2023

High order computation of optimal transport, mean field planning, and mean field games

Mean-field games (MFGs) have shown strong modeling capabilities for large systems in various fields, driving growth in computational methods for mean-field game problems. However, high order methods have not been thoroughly investigated. In this work, we explore applying general high-order numerical schemes with finite element methods in the space-time domain for computing the optimal transport (OT), mean-field planning (MFP), and MFG problems. We conduct several experiments to validate the convergence rate of the high order method numerically. Those numerical experiments also demonstrate the efficiency and effectiveness of our approach.

math.NA↗

arXiv2023

A kernel formula for regularized Wasserstein proximal operators

We study a class of regularized proximal operators in Wasserstein-2 space. We derive their solutions by kernel integration formulas. We obtain the Wasserstein proximal operator using a pair of forward-backward partial differential equations consisting of a continuity equation and a Hamilton-Jacobi equation with a terminal time potential function and an initial time density function. We regularize the PDE pair by adding forward and backward Laplacian operators. We apply Hopf-Cole type transformations to rewrite these regularized PDE pairs into forward-backward heat equations. We then use the fundamental solution of the heat equation to represent the regularized Wasserstein proximal with kernel integral formulas. Numerical examples show the effectiveness of kernel formulas in approximating the Wasserstein proximal operator.

math.OC↗

arXiv2022

A primal-dual approach for solving conservation laws with implicit in time approximations

In this work, we propose a novel framework for the numerical solution of time-dependent conservation laws with implicit schemes via primal-dual hybrid gradient methods. We solve an initial value problem (IVP) for the partial differential equation (PDE) by casting it as a saddle point of a min-max problem and using iterative optimization methods to find the saddle point. Our approach is flexible with the choice of both time and spatial discretization schemes. It benefits from the implicit structure and gains large regions of stability, and overcomes the restriction on the mesh size in time by explicit schemes from Courant--Friedrichs--Lewy (CFL) conditions (really via von Neumann stability analysis). Nevertheless, it is highly parallelizable and easy-to-implement. In particular, no nonlinear inversions are required! Specifically, we illustrate our approach using the finite difference scheme and discontinuous Galerkin method for the spatial scheme; backward Euler and backward differentiation formulas for implicit discretization in time. Numerical experiments illustrate the effectiveness and robustness of the approach. In future work, we will demonstrate that our idea of replacing an initial-value evolution equation with this primal-dual hybrid gradient approach has great advantages in many other situations.

math.NA↗

arXiv2022

Mean field control problems for vaccine distribution

With the invention of the COVID-19 vaccine, shipping and distributing are crucial in controlling the pandemic. In this paper, we build a mean-field variational problem in a spatial domain, which controls the propagation of pandemic by the optimal transportation strategy of vaccine distribution. Here we integrate the vaccine distribution into the mean-field SIR model designed in our previous paper arXiv:2006.01249. Numerical examples demonstrate that the proposed model provides practical strategies in vaccine distribution on a spatial domain.

math.OC↗