arXiv subjects

Namkyeong Cho

Publications and source records attributed to Namkyeong Cho.

8 recordsLinked to original sources

arXiv2026

Policy Iteration for Stationary Discounted Hamilton--Jacobi--Bellman Equations: A Viscosity Approach

We study policy iteration (PI) for deterministic infinite-horizon discounted control problems characterized by stationary Hamilton--Jacobi--Bellman equations. For general viscosity solutions, the classical gradient-based policy improvement step need not be defined pointwise. We introduce a semi-discrete formulation with centered difference quotients at scale $h$ and a separate artificial-viscosity term of order $O(h)$. The resulting stencil is monotone, and the positive discount yields a resolvent contraction. Under bounded Lipschitz data and a globally Lipschitz minimizing policy map, we prove monotone and geometric convergence of the value iterates for each fixed $h>0$, together with a local quadratic estimate whose constant is of order $h^{-2}$. Under the additional condition $λ>\Lip_x(f)$, we establish $\|V^h-V\|_\infty\le C\sqrt h$ and combine the discretization and iteration errors into a quantitative bound. A bounded Lipschitz example shows that the $\sqrt h$ exponent is sharp for this scheme. The combined estimate gives a sufficient iteration count of order $h^{-1}\log(1/h)$ to attain an error of order $\sqrt h$. In bounded-domain experiments, the smooth one-dimensional benchmark exhibits the predicted discretization plateau, while a nonlinear two-dimensional manufactured benchmark isolates convergence to the discrete solution. Exact policy evaluation gives substantially faster local convergence than the global geometric bound. A neural evaluation diagnostic illustrates the importance of controlling boundary errors as well as interior residuals.

math.OC↗

arXiv2026

Monotone Neural Policy Iteration for High-Dimensional First-Order Hamilton--Jacobi--Bellman Equations

We analyze a neural semi-discrete method for high-dimensional first-order Hamilton-Jacobi-Bellman (HJB) equations with known or learned dynamics. Centered differences and an artificial viscosity $Nh=O(h)$ define a monotone operator evaluated through $2d+1$ shifted network queries; policy iteration solves the resulting Bellman equation without a tensor grid. At fixed $h$, the sharp componentwise condition $\max_i|f_i|\le2N$ turns every frozen-policy operator into a nearest-neighbor Markov-chain generator with a policy-independent total jump rate. Uniformization gives whole-space well-posedness for measurable feedbacks, an explicit Poisson-tail bound on the numerical domain of dependence, and boundary-free localization. The representation also yields a posteriori policy-evaluation bounds that account for residual and learned-model errors. A greedy-gap analysis controls inexact policy iteration at fixed $h$; a separate consistency estimate connects the semi-discrete equation to the continuous HJB equation. Experiments reproduce the extremal tail, show rates consistent with $O(\sqrt h)$ and nearly $h$-independent exact-policy-iteration decay, and assess empirical estimator effectivity. A nonsmooth example shows that the continuous residual can miss a non-viscosity solution, whereas the shifted residual detects the defect. Further tests provide a structured interval-verified certificate calibration, an early-budget benefit of policy freezing for bang-bang control, and learned-dynamics diagnostics. A structured nonlinear problem with active compact-control constraints is tested against a manufactured semi-discrete reference through $d=1024$.

cs.LG↗

arXiv2026

Physics-Informed Policy Iteration for High-Dimensional Hamilton--Jacobi--Bellman Equations: Interior Error Bounds without Boundary Data

We develop a physics-informed policy-iteration method for stationary second-order Hamilton--Jacobi--Bellman equations arising in continuous-time stochastic control. Each policy-evaluation step is a linear elliptic PDE and is approximated by a mesh-free neural residual solver; policy improvement is then performed pointwise from the surrogate gradient. The analysis addresses bounded-domain training without prescribed boundary data. We prove well-posedness of the PDE-defined evaluation for every Borel Markov policy, establish Lipschitz stability of the greedy map on bounded gradient ranges, and derive an exponential attenuation estimate for unresolved boundary information. These ingredients yield a closed finite-step interior error bound whose floors are determined by a continuous $L^p$ residual, with a finite exponent $p>d$, and an attenuated amplitude term. The experiments measure the quantities in the estimate. On a linear--quadratic testbed with an exact reference, the gradient-error floor scales nearly linearly with a fresh-sample $L^p$ residual estimate, and finite-difference probes identify when a training buffer is beneficial. At matched architecture and budget, linear fixed-policy training becomes markedly more reliable than direct minimization of the nonlinear HJB residual as the tested problems become more difficult. The method also produces effective feedback on an inverted pendulum, a planar quadrotor, and a 100-dimensional posterior-seeking problem. These nonlinear tests expose two limitations not visible from sampled residuals alone: the collocation distribution may miss the region visited by the learned closed loop, and evaluation without boundary data may leave undetermined the gradient component on which the greedy update depends. On-policy collocation, rollout-anchored evaluation, and rollout-based stopping provide effective model-only safeguards.

cs.LG↗

arXiv2025

Physics-informed approach for exploratory Hamilton--Jacobi--Bellman equations via policy iterations

We propose a mesh-free policy iteration framework based on physics-informed neural networks (PINNs) for solving entropy-regularized stochastic control problems. The method iteratively alternates between soft policy evaluation and improvement using automatic differentiation and neural approximation, without relying on spatial discretization. We present a detailed $L^2$ error analysis that decomposes the total approximation error into three sources: iteration error, policy network error, and PDE residual error. The proposed algorithm is validated with a range of challenging control tasks, including high-dimensional linear-quadratic regulation in 5D and 10D, as well as nonlinear systems such as pendulum and cartpole problems. Numerical results confirm the scalability, accuracy, and robustness of our approach across both linear and nonlinear benchmarks.

math.NA↗

arXiv2024

Line Graph Vietoris-Rips Persistence Diagram for Topological Graph Representation Learning

While message passing graph neural networks result in informative node embeddings, they may suffer from describing the topological properties of graphs. To this end, node filtration has been widely used as an attempt to obtain the topological information of a graph using persistence diagrams. However, these attempts have faced the problem of losing node embedding information, which in turn prevents them from providing a more expressive graph representation. To tackle this issue, we shift our focus to edge filtration and introduce a novel edge filtration-based persistence diagram, named Topological Edge Diagram (TED), which is mathematically proven to preserve node embedding information as well as contain additional topological information. To implement TED, we propose a neural network based algorithm, named Line Graph Vietoris-Rips (LGVR) Persistence Diagram, that extracts edge information by transforming a graph into its line graph. Through LGVR, we propose two model frameworks that can be applied to any message passing GNNs, and prove that they are strictly more powerful than Weisfeiler-Lehman type colorings. Finally we empirically validate superior performance of our models on several graph classification and regression benchmarks.

cs.LG↗

arXiv2024

On the stability of Lipschitz continuous control problems and its application to reinforcement learning

We address the crucial yet underexplored stability properties of the Hamilton--Jacobi--Bellman (HJB) equation in model-free reinforcement learning contexts, specifically for Lipschitz continuous optimal control problems. We bridge the gap between Lipschitz continuous optimal control problems and classical optimal control problems in the viscosity solutions framework, offering new insights into the stability of the value function of Lipschitz continuous optimal control problems. By introducing structural assumptions on the dynamics and reward functions, we further study the rate of convergence of value functions. Moreover, we introduce a generalized framework for Lipschitz continuous control problems that incorporates the original problem and leverage it to propose a new HJB-based reinforcement learning algorithm. The stability properties and performance of the proposed method are tested with well-known benchmark examples in comparison with existing approaches.

math.OC↗

arXiv2024

Sobolev Training for Operator Learning

This study investigates the impact of Sobolev Training on operator learning frameworks for improving model performance. Our research reveals that integrating derivative information into the loss function enhances the training process, and we propose a novel framework to approximate derivatives on irregular meshes in operator learning. Our findings are supported by both experimental evidence and theoretical analysis. This demonstrates the effectiveness of Sobolev Training in approximating the solution operators between infinite-dimensional spaces.

cs.LG↗

arXiv2021

Global estimates of Generalized Non-Newtonian Stokes systems on non-smooth domains

We study a generalized Stokes system with Orlicz growth which is nonstandard in a non-smooth domain. Our purpose is to derive a Calderon-Zygmund type estimate of the gradient of a solution and the pressure to such a system like (1.1) under a small BMO non-linearity and a sufficient flatness on the boundary of the domain. In the process, we overcome not only lack of Lipschitz regularity for the corresponding limiting problem, but also the fact that the associated structure depends only on the symmetric part of the gradient for the desired global estimate, which is new even in the literature of elliptic system on a nonsmooth domain.

math.AP↗