arXiv subjects

James Chok

Publications and source records attributed to James Chok.

6 recordsLinked to original sources

arXiv2026

From Continuous Dynamics to Practical Gradient-Based Samplers

Gradient-based Markov chain Monte Carlo methods are often introduced as a catalog of algorithms: Hamiltonian Monte Carlo (HMC), the Metropolis-adjusted Langevin algorithm (MALA), the No-U-Turn Sampler (NUTS), and several underdamped variants. This presentation obscures the common structure of the methods and, more importantly, the reasons why a sampler that is correct in principle may be ineffective in practice. We develop a unified account, beginning with exact continuous-time dynamics that represent idealized sampling methods and for which Metropolis adjustments are not required. Numerical discretization makes the dynamics computationally feasible but introduces bias. Metropolis adjustment removes the asymptotic bias by converting numerical errors into rejection, leading to HMC, MALA, NUTS, and the Metropolis-adjusted kinetic Langevin algorithm (MAKLA). The second half of the paper presents geometric design choices that determine practical performance, namely, although MAKLA and NUTS have nice theoretical properties, their sampling efficiency may be slow in practice. Importantly, a fixed mass matrix can whiten globally anisotropic targets, often fixing sampling inefficiency in Bayesian posteriors with large data. Whereas hierarchical posteriors introduce their own problem, causing state-dependent variation in the Hessian (e.g., Neal's funnel). We explain how a randomized step size can be used effectively to sample from such a distribution. The resulting paper is both a tutorial on the mechanics of gradient-based sampling and a set of practical recipes to improve sampler performance.

stat.CO↗

arXiv2025

Constrained Dikin-Langevin diffusion for polyhedra

We propose a reflection-free Langevin framework for sampling and optimization on compact polyhedra. The method is based on the inverse Hessian of the logarithmic barrier, which defines a Dikin--Langevin diffusion whose drift and noise adapt to the local interior-point geometry. We show that trajectories started in the interior remain feasible for all finite times almost surely, so the constrained domain is preserved without reflections or projections. For computation, we discretize the diffusion using the Euler--Maruyama scheme and apply a Metropolis--Hastings correction, yielding a sampler that targets the exact constrained distribution. We also propose an annealed interacting variant for nonconvex optimization. Numerically, the Metropolis-adjusted method outperforms both the Dikin random walk and standard MALA on anisotropic box-constrained Gaussians, and the interacting optimizer escapes suboptimal basins more reliably than the non-interacting method.

stat.CO↗

arXiv2025

Divide, Interact, Sample: The Two-System Paradigm

Mean-field, ensemble-chain, and adaptive samplers have historically been viewed as distinct approaches to Monte Carlo sampling. In this paper, we present a unifying {two-system} framework that brings all three under one roof. In our approach, an ensemble of particles is split into two interacting subsystems that propose updates for each other in a symmetric, alternating fashion. For the memoryless two-system samplers, this cross-system interaction ensures that the finite ensemble has $\rho^{\otimes 2N}$ as its invariant distribution; for finite-adaptive variants, exact stationarity applies after the adaptation phase is frozen. The two-system construction reveals that ensemble-chain samplers can be interpreted as finite-$N$ approximations to an ideal mean-field sampler; conversely, it provides a principled recipe for discretizing mean-field Langevin dynamics into tractable parallel MCMC algorithms. The framework also connects naturally to adaptive single-chain methods: by replacing particle-based statistics with time-averaged statistics from a single chain, one recovers analogous adaptive dynamics in the long-time limit without requiring a large ensemble. We derive novel two-system versions of both overdamped and underdamped Langevin MCMC samplers within this paradigm. Across synthetic benchmarks and real-world posterior inference tasks, these two-system samplers -- which use a single BCSS-2 integrator step per Metropolis--Hastings accept/reject, in contrast to the long-trajectory style of HMC/NUTS -- exhibit substantial performance gains over No-U-Turn Sampler baselines, achieving higher effective sample sizes per gradient evaluation and markedly higher wall-clock throughput. On higher-dimensional posteriors, the adaptive MAKLA-BCSS-2 methods remain stable and achieve substantially better per-gradient efficiency and wall-clock throughput than the NUTS variants in our benchmark suite.

stat.CO↗

arXiv2023

Rational function approximation with normalized positive denominators

Recent years have witnessed the introduction and development of extremely fast rational function algorithms. Many ideas in this realm arose from polynomial-based linear-algebraic algorithms. However, polynomial approximation is occasionally ill-suited to specific challenging tasks arising in several situations. Some occasions require maximal efficiency in the number of encoding parameters whilst retaining the renowned accuracy of polynomial-based approximation. One application comes from promoting empirical pointwise functions to sparse matrix operators. Rational function approximations provide a simple but flexible alternative (actually a superset), allowing one to capture complex non-linearities. However, these come with extra challenges: i) coping with singularities and near singularities arising from a vanishing denominator, and ii) a non-uniqueness owing to a simultaneous renormalization of both numerator and denominator. We, therefore, introduce a new rational function framework using manifestly positive and normalized Bernstein polynomials for the denominator and any traditional polynomial basis (e.g., Chebyshev) for the numerator. While an expressly non-singular approximation slightly reduces the maximum degree of compression, it keeps all the benefits of rational functions while maintaining the flexibility and robustness of polynomials. We illustrate the relevant aspects of this approach with a series of derivations and computational examples.

math.NA↗

arXiv2023

Convex optimization over a probability simplex

We propose a new iteration scheme, the Cauchy-Simplex, to optimize convex problems over the probability simplex $\{w\in\mathbb{R}^n\ |\ \sum_i w_i=1\ \textrm{and}\ w_i\geq0\}$. Specifically, we map the simplex to the positive quadrant of a unit sphere, envisage gradient descent in latent variables, and map the result back in a way that only depends on the simplex variable. Moreover, proving rigorous convergence results in this formulation leads inherently to tools from information theory (e.g., cross-entropy and KL divergence). Each iteration of the Cauchy-Simplex consists of simple operations, making it well-suited for high-dimensional problems. In continuous time, we prove that $f(x_T)-f(x^*) = {O}(1/T)$ for differentiable real-valued convex functions, where $T$ is the number of time steps and $w^*$ is the optimal solution. Numerical experiments of projection onto convex hulls show faster convergence than similar algorithms. Finally, we apply our algorithm to online learning problems and prove the convergence of the average regret for (1) Prediction with expert advice and (2) Universal Portfolios.

math.OC↗

arXiv2023

Geometric persistence and distributional trends in worldwide terrorism

This paper introduces new methods for studying the prevalence of terrorism around the world and over time. Our analysis treats spatial prevalence of terrorism, the changing profile of groups carrying out the acts of terrorism, and trends in how many attacks take place over time. First, we use a time-evolving cluster analysis to show that the geographic distribution of regions of high terrorist activity remains relatively consistent over time. Secondly, we use new metrics, inspired by geometry and probability, to track changes in the distributions of which groups are performing the terrorism. We identify times at which this distribution changes significantly and countries where the time-varying breakdown is most and least homogeneous. We observe startling geographic patterns, with the greatest heterogeneity from Africa. Finally, we use a new implementation of distances between distributions to group countries according to their incidence profiles over time. This analysis can aid in highlighting structural similarities in outbreaks of extreme behavior and the most and least significant public policies in minimizing a country's terrorism.

physics.soc-ph↗