arXiv subjects

Samy Wu Fung

Publications and source records attributed to Samy Wu Fung.

At least 19 recordsLinked to original sources

arXiv2026

Higher-Order Mean-Field Control Barrier Functions

Mean-field control barrier functions (MF-CBFs) enforce swarm safety through constraints on the agents' distribution. Previous formulations, including applications to stochastic coverage and shepherding, use first-order differential inequalities. These cannot directly enforce constraints whose barrier functions yield control-free first derivatives. Thus, we develop the theory of higher-order MF-CBFs, which ensures the positivity of the barrier functionals through higher-order differential inequalities, as in their finite-dimensional analogs. More specifically, we introduce the notion of relative degree for mean-field safety functionals and develop an analytic framework for computing their higher-order derivatives. Moreover, for practical examples of cross-correlation and self-correlation functionals, we show that repeated differentiation along mean-field dynamics is structure-preserving and reduces to kernel recursions. As an illustration, we apply higher-order MF-CBFs to double-integrator swarm tracking and avoidance problems, where position-only constraints lead to MF-CBFs of relative degree two.

math.OC↗

arXiv2026

Truncated Differentiation Through Primal-Dual Solvers for Inverse Potential Mean-Field Games

We study inverse potential mean-field games (MFGs), in which an unknown spatial inverse-cost (mobility) map is inferred from observed population densities. We solve the forward MFG with a preconditioned primal-dual hybrid gradient (PDHG) method and develop Jacobian-free backpropagation (JFB-$r$), which records only the final $r$ iterations from a detached warm start while retaining the full forward solve. To analyze this truncated differentiation method, we show that the exact-proximal dual-extrapolated PDHG map is a metric resolvent of the maximal monotone KKT operator. This resolvent view shows that JFB-$r$ exactly differentiates a finite-trajectory surrogate and, under a locally fixed active set at an exact equilibrium detach point, converges to the implicit gradient as the tracked depth increases. Across several inverse-MFG settings, numerical experiments show that JFB-r at moderate tracked depths can achieve recovery accuracy comparable to full unrolling while reducing memory and runtime.

math.OC↗

arXiv2026

End-to-End Learning of Safe Optimal Feedback Control in High Dimensions with Control Barrier Function Layers

We consider the problem of learning high-dimensional semi-global feedback controllers under hard safety constraints enforced by control barrier functions (CBFs). Incorporating CBFs into end-to-end policy training requires embedding a quadratic-program-based safety filter as an optimization layer, but computational and differentiation bottlenecks have largely restricted prior approaches to low-dimensional systems, typically with at most 16 state dimensions. We address this limitation by combining operator splitting with the recently developed Jacobian-Free Backpropagation (JFB) method to enable scalable end-to-end training while preserving hard safety guarantees through the CBF safety filter. We justify this training methodology theoretically using nonsmooth analysis techniques and demonstrate its effectiveness on high-dimensional multi-agent nonlinear control problems with state and control dimensions up to 1200 and 400, respectively.

cs.LG↗

arXiv2026

Implicit Neural Optimal Transport via Fixed-Point Optimization

We propose an implicit neural formulation of optimal transport that eliminates adversarial min--max optimization and multi-network architectures commonly used in existing approaches. Our key idea is to parameterize a single potential in the Kantorovich dual and reformulate the associated c-transform as a proximal fixed-point problem. This yields a stable single-network framework in which dual feasibility is enforced exactly through proximal optimality conditions rather than adversarial training. Despite the inner fixed-point computation, gradients can be computed without differentiating through the fixed-point iterations, enabling efficient training without requiring implicit differentiation. We further establish convergence of stochastic gradient descent. The resulting framework is efficient, scalable, and broadly applicable: it simultaneously recovers forward and backward transport maps and naturally extends to class-conditional settings. Experiments on high-dimensional Gaussian benchmarks, physical datasets, and image translation tasks demonstrate strong transport accuracy together with improved training stability and favorable computational and memory efficiency.

math.OC↗

arXiv2026

Probabilistic Gaussian Homotopy: A Probability-Space Continuation Framework for Nonconvex Optimization

We introduce Probabilistic Gaussian Homotopy (PGH), a probability-space continuation framework for nonconvex optimization. Unlike classical Gaussian homotopy, which smooths the objective and uniformly averages gradients, PGH deforms the associated Boltzmann distribution and induces Boltzmann-weighted aggregation of perturbed gradients, which exponentially biases descent directions toward low-energy regions. We show that PGH corresponds to a log-sum-exp (soft-min) homotopy that smooths a nonconvex objective at scale $λ>0$ and recovers the original objective as $λ\to 0$, yielding a posterior-mean generalization of the Moreau envelope, and we derive a dynamical system governing minimizer evolution along an annealed homotopy path. This establishes a principled connection between Gaussian continuation, Bayesian denoising, and diffusion-style smoothing. We further propose Probabilistic Gaussian Homotopy Optimization (PGHO), a practical stochastic algorithm based on Monte Carlo gradient estimation, and demonstrate strong performance on high-dimensional nonconvex benchmarks and sparse recovery problems where classical gradient methods and objective-space smoothing frequently fail.

cs.LG↗

arXiv2026

Operator Splitting with Hamilton-Jacobi-based Proximals

Operator splitting algorithms are a cornerstone of modern first-order optimization, decomposing complex problems into simpler subproblems solved via proximal operators. However, most functions lack closed-form proximal operators, which has long restricted these methods to a narrow set of problems. Hamilton-Jacobi-based proximal operator (HJ-Prox) is a recent derivative-free Monte Carlo technique based on Hamilton-Jacobi PDE theory, that approximates proximal operators numerically. In this work, we introduce a unified framework for operator splitting via HJ-Prox, which allows for deployment of operator splitting even when functions are not proximable. We prove that replacing exact proximal steps with HJ-Prox in algorithms such as proximal point, proximal gradient descent, Douglas-Rachford splitting, Davis-Yin splitting, and primal-dual hybrid gradient preserves convergence guarantees under mild assumptions. Numerical experiments demonstrate HJ-Prox is competitive and effective on a wide variety of statistical learning tasks.

math.OC↗

arXiv2026

On the Convergence of Jacobian-Free Backpropagation for Optimal Control Problems with Implicit Hamiltonians

Optimal feedback control with implicit Hamiltonians poses a fundamental challenge for learning-based value function methods due to the absence of closed-form optimal control laws. Recent work~\cite{gelphman2025end} introduced an implicit deep learning approach using Jacobian-Free Backpropagation (JFB) to address this setting, but only established sample-wise descent guarantees. In this paper, we establish convergence guarantees for JFB in the stochastic minibatch setting, showing that the resulting updates converge to stationary points of the expected optimal control objective. We further demonstrate scalability on substantially higher-dimensional problems, including multi-agent optimal consumption and swarm-based quadrotor and bicycle control. Together, our results provide both theoretical justification and empirical evidence for using JFB in high-dimensional optimal control with implicit Hamiltonians.

math.OC↗

arXiv2026

Mean-field control barrier functions for stochastic multi-agent systems

Many applications involving multi-agent systems require fulfilling safety constraints. Control barrier functions offer a systematic framework to enforce forward invariance of safety sets. Recent work extended this paradigm to mean-field scenarios, where the number of agents is large enough to make density-space descriptions a reasonable workaround for the curse of dimensionality. However, an open gap in the recent literature concerns the development of mean-field control barrier functions for Fokker-Planck (advection-diffusion) equations. In this work, we address this gap, enabling safe mean-field control of agents with stochastic microscopic dynamics. We provide bounded stability guarantees under safety corrections and corroborate our results through numerical simulations in two representative scenarios, coverage and shepherding control of multi-agent systems.

eess.SY↗

arXiv2026

A Gauss-Newton Method with No Additional PDE Solves Beyond Gradient Evaluation for Large-Scale PDE-Constrained Inverse Problems

Partial Differential Equation (PDE)-constrained optimization problems often take the form of an optimization of an objective function given as a sum of loss terms. Each function or gradient evaluation requires one or more PDE solves, which render these problems computationally demanding. While Gauss-Newton methods are well-suited for large-scale PDE-constrained optimization, their application to settings such as Full-Waveform Inversion (FWI) is hindered by the need for additional PDE solves to compute Jacobian-vector products. This paper proposes a Gauss-Newton approach that eliminates the need for extra PDE solves beyond those required for gradient computation. Our numerical experiments on FWI demonstrate that the proposed method achieves the efficiency of gradient-based schemes while retaining the fast convergence of Gauss-Newton methods.

math.OC↗

arXiv2026

A Generalization Bound for a Family of Implicit Networks

Implicit networks are a class of neural networks whose outputs are defined by the fixed point of a parameterized operator. They have enjoyed success in many applications including natural language processing, image processing, and numerous other applications. While they have found abundant empirical success, theoretical work on its generalization is still under-explored. In this work, we consider a large family of implicit networks defined parameterized contractive fixed point operators. We show a generalization bound for this class based on a covering number argument for the Rademacher complexity of these architectures.

cs.LG↗

arXiv2025

Recent Advances in Numerical Solutions for Hamilton-Jacobi PDEs

Hamilton-Jacobi partial differential equations (HJ PDEs) play a central role in many applications such as economics, physics, and engineering. These equations describe the evolution of a value function which encodes valuable information about the system, such as action, cost, or level sets of a dynamic process. Their importance lies in their ability to model diverse phenomena, ranging from the propagation of fronts in computational physics to optimal decision-making in control systems. This paper provides a review of some recent advances in numerical methods to address challenges such as high-dimensionality, nonlinearity, and computational efficiency. By examining these developments, this paper sheds light on important techniques and emerging directions in the numerical solution of HJ PDEs.

math.NA↗

arXiv2025

End-to-End Training of High-Dimensional Optimal Control with Implicit Hamiltonians via Jacobian-Free Backpropagation

Neural network approaches that parameterize value functions have succeeded in approximating high-dimensional optimal feedback controllers when the Hamiltonian admits explicit formulas. However, many practical problems, such as the space shuttle reentry problem and bicycle dynamics, among others, may involve implicit Hamiltonians that do not admit explicit formulas, limiting the applicability of existing methods. Rather than directly parameterizing controls, which does not leverage the Hamiltonian's underlying structure, we propose an end-to-end implicit deep learning approach that directly parameterizes the value function to learn optimal control laws. Our method enforces physical principles by ensuring trained networks adhere to the control laws by exploiting the fundamental relationship between the optimal control and the value function's gradient; this is a direct consequence of the connection between Pontryagin's Maximum Principle and dynamic programming. Using Jacobian-Free Backpropagation (JFB), we achieve efficient training despite temporal coupling in trajectory optimization. We show that JFB produces descent directions for the optimal control objective and experimentally demonstrate that our approach effectively learns high-dimensional feedback controllers across multiple scenarios involving implicit Hamiltonians, which existing methods cannot address.

math.OC↗

arXiv2025

A primal-dual price-optimization method for computing equilibrium prices in mean-field games models

We develop a simple yet efficient Lagrangian method for computing equilibrium prices in a mean-field game price-formation model. We prove that equilibrium prices are optimal in terms of a suitable criterion and derive a primal-dual gradient-based algorithm for computing them. One of the highlights of our computational framework is the efficient, simple, and flexible implementation of the algorithm using modern automatic differentiation techniques. Our implementation is modular and admits a seamless extension to high-dimensional settings with more complex dynamics, costs, and equilibrium conditions. Additionally, automatic differentiation enables a versatile algorithm that requires only coding the cost functions of agents. It automatically handles the gradients of the costs, thereby eliminating the need to manually form the adjoint equations.

math.OC↗

arXiv2025

A Monte Carlo Approach for Nonsmooth Convex Optimization via Proximal Splitting Algorithms

Operator splitting algorithms are a cornerstone of modern first-order optimization, relying critically on proximal operators as their fundamental building blocks. However, explicit formulas for proximal operators are available only for limited classes of functions, restricting the applicability of these methods. Recent work introduced HJ-Prox, a zeroth-order Monte Carlo approximation of the proximal operator derived from Hamilton-Jacobi PDEs, which circumvents the need for closed-form solutions. In this work, we extend the scope of HJ-Prox by establishing that it can be seamlessly incorporated into operator splitting schemes while preserving convergence guarantees. In particular, we show that replacing exact proximal steps with HJ-Prox approximations in algorithms such as proximal gradient descent, Douglas-Rachford splitting, Davis-Yin splitting, and the primal-dual hybrid gradient method still ensures convergence under mild conditions.

math.OC↗

arXiv2025

On Logical Extrapolation for Mazes with Recurrent and Implicit Networks

Recent work suggests that certain neural network architectures -- particularly recurrent neural networks (RNNs) and implicit neural networks (INNs) -- are capable of logical extrapolation. When trained on easy instances of a task, these networks (henceforth: logical extrapolators) can generalize to more difficult instances. Previous research has hypothesized that logical extrapolators do so by learning a scalable, iterative algorithm for the given task which converges to the solution. We examine this idea more closely in the context of a single task: maze solving. By varying test data along multiple axes -- not just maze size -- we show that models introduced in prior work fail in a variety of ways, some expected and others less so. It remains uncertain whether any of these models has truly learned an algorithm. However, we provide evidence that a certain RNN has approximately learned a form of `deadend-filling'. We show that training these models on more diverse data addresses some failure modes but, paradoxically, does not improve logical extrapolation. We also analyze convergence behavior, and show that models explicitly trained to converge to a fixed point are likely to do so when extrapolating, while models that are not may exhibit more exotic limiting behavior such as limit cycles, even when they correctly solve the problem. Our results (i) show that logical extrapolation is not immune to the problem of goal misgeneralization, and (ii) suggest that analyzing the dynamics of extrapolation may yield insights into designing better logical extrapolators.

cs.LG↗

arXiv2025

Thinking Out of the Box: Hybrid SAT Solving by Unconstrained Continuous Optimization

The Boolean satisfiability (SAT) problem lies at the core of many applications in combinatorial optimization, software verification, cryptography, and machine learning. While state-of-the-art solvers have demonstrated high efficiency in handling conjunctive normal form (CNF) formulas, numerous applications require non-CNF (hybrid) constraints, such as XOR, cardinality, and Not-All-Equal constraints. Recent work leverages polynomial representations to represent such hybrid constraints, but it relies on box constraints that can limit the use of powerful unconstrained optimizers. In this paper, we propose unconstrained continuous optimization formulations for hybrid SAT solving by penalty terms. We provide theoretical insights into when these penalty terms are necessary and demonstrate empirically that unconstrained optimizers (e.g., Adam) can enhance SAT solving on hybrid benchmarks. Our results highlight the potential of combining continuous optimization and machine-learning-based methods for effective hybrid SAT solving.

cs.LO↗

arXiv2025

A hybrid SIAC -- data-driven post-processing filter for discontinuities in solutions to numerical PDEs

We present a hybrid filter that is only applied to the approximation at the final time and allows for reducing errors away from a shock as well as near a shock. It is designed for discontinuous Galerkin approximations to PDEs and combines a rigorous moment-based Smoothness-Increasing Accuracy-Conserving (SIAC) filter with a data-driven CNN filter. While SIAC improves accuracy in smooth regions, it fails to reduce the $\mathcal{O}(1)$ errors near discontinuities, particularly in inviscid compressible flows with shocks. Our hybrid SIAC-CNN filter, trained exclusively on top-hat functions, enforces consistency constraints globally and higher-order moment conditions in smooth regions, reducing both $\ell_2$ and $\ell_\infty$ errors near discontinuities and preserving theoretical accuracy in smooth regions. We demonstrate its effectiveness on the Euler equations for the Lax, Sod, and Shu-Osher shock-tube problems.

math.NA↗

arXiv2024

Mean-Field Control Barrier Functions: A Framework for Real-Time Swarm Control

Control Barrier Functions (CBFs) are an effective methodology to ensure safety and performative efficacy in real-time control applications such as power systems, resource allocation, autonomous vehicles, robotics, etc. This approach ensures safety independently of the high-level tasks that may have been pre-planned offline. For example, CBFs can be used to guarantee that a vehicle will remain in its lane. However, when the number of agents is large, computation of CBFs can suffer from the curse of dimensionality in the multi-agent setting. In this work, we present Mean-field Control Barrier Functions (MF-CBFs), which extends the CBF framework to the mean-field (or swarm control) setting. The core idea is to model a population of agents as probability measures in the state space and build corresponding control barrier functions. Similar to traditional CBFs, we derive safety constraints on the (distributed) controls but now relying on the differential calculus in the space of probability measures.

math.OC↗