Searcharxiv⌕ Search

SEARCH · Searcharxiv

Search Searcharxiv

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8Linked to original sources
arXiv2026

Gradient Descent on Logistic Regression with Non-Separable Data and Large Step Sizes

We study gradient descent (GD) dynamics on logistic regression problems with large, constant step sizes. For linearly-separable data, it is known that GD converges to the minimizer with arbitrarily large step sizes, a property which no longer holds when the problem is not separable. In fact, the behaviour can be much more complex -- a sequence of period-doubling bifurcations begins at the critical step size $2/λ$, where $λ$ is the largest eigenvalue of the Hessian at the solution. Using a smaller-than-critical step size guarantees convergence if initialized nearby the solution: but does this suffice globally? In one dimension, we show that a step size less than $1/λ$ suffices for global convergence. However, for all step sizes between $1/λ$ and the critical step size $2/λ$, one can construct a dataset such that GD converges to a stable cycle. In higher dimensions, this is actually possible even for step sizes less than $1/λ$. Our results show that although local convergence is guaranteed for all step sizes less than the critical step size, global convergence is not, and GD may instead converge to a cycle depending on the initialization.

cs.LG↗
arXiv2026

Optimal Sensor and Actuator Selection for Factored Markov Decision Processes: Complexity, Approximability and Algorithms

Factored Markov Decision Processes (fMDPs) are a class of Markov Decision Processes (MDPs) in which the states (and actions) can be factored into a set of state (and action) variables and can be encoded compactly using a factored representation. In this paper, we consider a setting where the state of the fMDP is not directly observable, and the agent relies on a set of potential sensors to gather information. We formulate the problem of selecting a set of sensors for fMDPs (under a limited budget) to maximize the infinite-horizon discounted return provided by the optimal policy. We show the fundamental result that it is NP-hard to approximate this problem to within a factor of $n^{1-c}$ for any $c > 1$, where $n$ is the number of state variables. Our inapproximability results for sensor selection also extend to a general class of Partially Observable MDPs (POMDPs). We also consider the dual problem of budgeted actuator selection (at design-time) to maximize the expected return under the optimal policy, for which we establish similar inapproximability results. Finally, we consider a simple greedy algorithm and empirically show that, despite the lack of formal theoretical guarantees, it performs effectively in practice, achieving on average over $70\%$ of the optimal solution value across a variety of real-world and randomly generated problem instances.

eess.SY↗
arXiv2026

Sample Complexity of Linear Quadratic Regulator Without Initial Stability

Inspired by REINFORCE, we introduce a novel receding-horizon algorithm for the Linear Quadratic Regulator (LQR) problem with unknown dynamics. Unlike prior methods, our algorithm avoids reliance on two-point gradient estimates while maintaining the same order of sample complexity. Furthermore, it eliminates the restrictive requirement of starting with a stable initial policy, broadening its applicability. Beyond these improvements, we introduce a refined analysis of error propagation through the contraction of the Riccati operator under the Riemannian distance. This refinement leads to a better sample complexity and ensures improved convergence guarantees.

math.OC↗
arXiv2026

Quantum Speedups for Sampling and Non-convex Optimization with Stochastic Oracles

We present quantum speedups for sampling from distributions of the form $π\propto e^{-f}$ on $\mathbb{R}^d$. We consider two stochastic oracle models: a stochastic gradient oracle, where $f=\frac{1}{n}\sum_{i=1}^n f_i $ and component gradients $\{\nabla f_i\}_{i \in [n]}$ are available, and a stochastic evaluation oracle, where only noisy values of $f$ are available. Our framework accelerates classical stochastic Langevin Monte Carlo (LMC) and Hamiltonian Monte Carlo (HMC) algorithms by replacing stochastic gradient estimators with variance-controlled quantum mean estimation and gradient estimation subroutines. Unlike quantum walk based approaches, our algorithms do not require reversibility or exact gradients, and they preserve the structure of the underlying Markov chain. In the finite-sum setting, quantum mean estimation combined with classical variance-reduction techniques improves the stochastic gradient-query complexity for the approximate sampling task. In the stochastic zeroth-order setting, we develop gradient estimators robust to noisy function evaluations, yielding improved evaluation complexity for LMC and HMC. These results apply to strongly log-concave and/or non-log-concave distributions satisfying a log-Sobolev inequality, with convergence guarantees in Wasserstein distance and Kullback--Leibler divergence. We also show that faster sampling methods lead to quantum speedups for optimization, including for non-smooth and approximately convex objectives.

quant-ph↗
arXiv2026

An Energy-Based Mechanism for Compositional Behavior

Flexible intelligence relies on the ability to reuse previously acquired behaviors and combine them differently as circumstances change. In biological and artificial systems, this ability is often attributed to gating mechanisms that determine how much each available behavior should contribute at a given time. Yet these gating rules, the dynamics that compute them, and the neural circuits that may implement them are usually introduced separately, leaving unclear whether they reflect a common underlying principle. Here, we show that they can all be derived from a single variational principle for behavioral composition. The resulting mechanism naturally gives rise to softmax gating, evolves as an energy-based dynamical system with guaranteed convergence, and admits a recurrent neural network instantiation featuring context-dependent and local interactions. Across collective behavior, human decision-making, and layered control, the same mechanism reproduces characteristic behavioral patterns, provides interpretable accounts of how different behaviors are combined, and matches or outperforms established approaches. These results provide a unified account of how behavioral composition can emerge from a common principle, with implications for understanding flexible behavior in natural systems and for designing artificial agents that can adapt by recombining existing capabilities.

math.OC↗
arXiv2026

Constrained dynamics for searching saddle points on embedded Riemannian submanifolds of Euclidean space

Finding constrained saddle points on embedded Riemannian submanifolds of Euclidean space is significant for analyzing energy landscapes arising in physics and chemistry. Existing works exploit explicit global/local regular level-set representations of manifolds, which may be unavailable or computationally inconvenient for manifolds represented through, e.g., projectors, factorizations, or rank constraints. In this paper, we develop a constrained saddle dynamic based on embedded-submanifold geometric primitives, completely avoiding the use of explicit representations. In particular, our dynamic is formulated compactly on the Grassmann bundle of the tangent bundle. By analyzing the Grassmann bundle geometry, we rigorously establish the local linear stability of the dynamic and the local linear convergence of the resulting algorithms. Remarkably, our analysis provides the first iterate convergence result for discretized algorithms to saddle points of prescribed indices in embedded-submanifold settings. Moreover, by virtue of the Grassmann bundle formulation, we remove unnecessary nondegeneracy assumptions on the eigenvalues of the Riemannian Hessian that are present in existing works. We also point out that locating saddle points can be more ill-conditioned than finding local minimizers, and requires using nonredundant parametrizations. Finally, numerical experiments on linear eigenvalue problems and electronic excited-state calculations showcase the effectiveness of the proposed algorithms and corroborate the established local theory.

math.NA↗
arXiv2026

Trajectory-Wise Certification for Vector-Field Mirror Descent with Deterministic Finite Differences

We study mirror descent in which the objective gradient is replaced by a general vector field. Since such a field does not automatically relate the mirror update to the objective gap, we introduce a trajectory-wise generalized relative-smoothness condition and a generalized star-convexity interface. Together they yield a finite-horizon, a posteriori last-iterate certificate with an accumulated-stepsize term and an exceptional-region-dependent error term. We also give a pointwise sufficient condition for positive admissible stepsizes, displaying the effects of objective smoothness, mirror-map conditioning, and vector-field mismatch. We then construct a deterministic zeroth-order instance from coordinate central differences. The function values place the unknown gradient in an explicit uncertainty box, and verification of the interface reduces to robust conic dominance. We derive an explicit scaling of the central-difference vector that guarantees the required dominance over the gradient uncertainty set. Under uniform Hessian bounds, the resulting vector field satisfies the interface outside an explicit resolution-dependent neighborhood. The final guarantee combines a rate term governed by accumulated accepted stepsizes with a finite-resolution error floor.

math.OC↗
arXiv2026

Variational Contraction Conditions for Iterative Algorithms in Multi-Population Discrete-Time Regularized Mean-Field Games

In this work, we study the contraction conditions of iterative algorithms for stationary and finite-horizon discrete-time regularized mean-field games (MFGs) with multiple populations, where each population only interacts with the state distributions of the other populations. Due to the high dimensionality caused by the interaction of different populations, contraction rates for these algorithms cannot, in general, be expressed in terms of radicals. By studying the dynamics of these iterative algorithms and assuming that the system components of each population's MFG are Lipschitz continuous, we present explicit (eventual) contraction conditions for each algorithm in any normed space, relying only on these Lipschitz parameters. As a consequence of these contraction conditions, we provide convergence rates of finite-horizon mean-field equilibria to infinite-horizon stationary (and non-stationary) mean-field equilibria (MFEs), under restrictions on a variational characterization of the dynamics of these iterative algorithms. In the single-population case, the restrictions we impose on this variational characterization to obtain these convergence results are less restrictive than previous results in the literature.

math.OC↗
arXiv2026

Scalable Self-Supervised Learning for Multiphase AC-OPF in Distribution Systems with Topology Reconfiguration

The proliferation of distributed energy resources (DERs) in distribution grids enables the active coordination of these assets to reduce costs and enable cleaner operations. Realizing this potential requires solving multiphase AC optimal power flow (AC-OPF) quickly across varying loads, DER availabilities, and topology reconfigurations, at much greater speed and scale than conventional nonlinear solvers. Learning-based surrogates can offer millisecond inference, yet existing methods target largely balanced transmission systems and do not scale to the multiphase, unbalanced, and reconfigurable nature of distribution feeders at utility scale. We present the Penalty + Sequential Linearized Feasibility Seeking (SLFS) algorithm, a self-supervised learning framework for multiphase distribution AC-OPF under switch-induced topology changes. Penalty+SLFS requires no labeled optimal solutions and trains directly from the AC-OPF objective and constraints through a differentiable fixed-point power flow solver, avoiding expensive label generation and admitting robust training procedures. Topology changes are handled efficiently using Sherman-Morrison-Woodbury updates of the admittance-matrix inverse, while an M-step Jacobian approximation accelerates differentiation through the power flow solver. At inference, SLFS repairs any infeasible predictions, providing feasibility guarantees with low computational overhead. On IEEE feeders ranging from 13 to 8,500 nodes, Penalty+SLFS achieves negligible optimality gaps and near-zero constraint violations, delivers up to three orders of magnitude speedups over IPOPT, and remains robust under large distributional shifts, demonstrating a viable path toward real-time, topology-aware AC-OPF for large-scale distribution grids.

eess.SY↗
arXiv2026

Turnpike properties in nonlinear system identification

We analyze the problem of learning general discrete-time nonlinear state-space models using the simulation error minimization (SEM) method. In this setting, model parameters are typically learned by minimizing the mismatch between simulated and measured outputs over a training dataset, or shorter subsequences extracted from it. Specifically, we study the cumulative output turnpike property of the underlying SEM optimization problem, which requires optimal output sequences emanating from a fixed initial state to approach and remain close to an optimal output sequence of the corresponding SEM problem with free initial state. In the presence of non-unique optimal output sequences---as may arise, for instance, in fully black-box system identification using neural networks---the property is formulated with respect to the closest such sequence. Turnpike behavior is generally desirable in practice, as it provides a theoretical justification for employing computationally more tractable SEM formulations with fixed initial states while ensuring that their optimal output sequences remain close to unconstrained optimal ones. Under a mild reachability condition, we establish equivalence between the cumulative turnpike property, coercivity of the value function, and a tailored notion of strict dissipativity. We additionally introduce a cardinality turnpike property and show that it is strictly weaker than the cumulative notion. Finally, we establish sufficient conditions for turnpike behavior based on incremental output stability, convexity of the stage cost, and a suitable optimality condition, and illustrate the theory by means of a numerical example.

eess.SY↗
arXiv2026

Two Adjoint Perspectives on Fokker-Planck Optimization: A Microscopic-Macroscopic Correspondence

The Fokker-Planck equation admits both a macroscopic Eulerian description through probability densities and a microscopic Lagrangian description through stochastic trajectories. Consequently, optimization problems constrained by the Fokker-Planck equation can be formulated from either perspective. Surprisingly, the corresponding adjoint equations appear to be fundamentally different: the macroscopic adjoint is governed by the backward Kolmogorov equation, whereas the microscopic adjoint evolves pathwise along stochastic trajectories. In this note, we reconcile these two formulations by establishing their correspondence in the continuum setting. We further show that, although their discrete gradients no longer coincide after discretization, both provide consistent numerical approximations of the continuum gradient. Explicit convergence rates are established for both discretization strategies.

math.NA↗
arXiv2026

Characterizations and Complexity of Minimum Forward and Integer Cycle Bases

The cycle space of a directed graph is generated by a cycle basis, where, in general, cycles are allowed to have both forward and backward arcs. In a forward cycle, all arcs must follow the given direction. Several open questions remain regarding the complexity of the minimum cycle basis problem, in particular the minimum-weight integral cycle basis problem, and the minimum-weight weakly and strictly fundamental forward cycle basis problems. In this paper, we address these open questions. First, we study the existence, structure, and computational complexity of minimum-weight forward cycle bases. We give a complete structural characterization of digraphs that admit weakly fundamental (and hence integral) forward cycle bases. We further provide a characterization when a strongly connected digraph admits a forward fundamental cycle basis, proving that such a basis exists if and only if the set of directed cycles has cardinality equal to the cycle rank; in this case, the basis is unique. Lastly, we show that while minimum-weight forward fundamental cycle bases can be found in polynomial time whenever they exist, the minimum-weight forward weakly fundamental cycle basis problem is APX-hard via an L-reduction from the minimum-weight weakly fundamental cycle basis problem on digraphs with metric weights. Second, we introduce opt-in graphs, i.e., the family of graphs for which minimum cycle bases are integral for any weight function. We show that this family is minor-closed and hence, by the Robertson-Seymour theorem, is characterized by a finite set of forbidden minors, so that the opt-in recognition problem is solvable in polynomial time. Lastly, we present an algorithm to check whether a graph is opt-in, and if not, to identify which of its minors belong to the set of forbidden minors. Applying this algorithm, we show that the complete graph $K_n$ is opt-in if and only if $n \leq 7$.

math.OC↗
arXiv2026

A simple derivation of the Kalman filter

In this lecture note, we present a concise and self-contained derivation of the discrete-time Kalman filter equations that requires only a basic understanding of least squares estimation. The treatment is designed to minimize mathematical overhead while preserving both rigor and generality.

math.OC↗
arXiv2026

On the invariance of risk-sensitive LQR gain under input randomization

This paper shows that the optimal gain of the risk-sensitive linear quadratic regulator (LQR) problem is invariant under input randomization, i.e., when the controller deliberately injects noise into the nominal control input. This appears counterintuitive at first glance because certainty equivalence does not hold for risk-sensitive LQR and input randomization inflates the effective process noise. Nonetheless, the gain is preserved because the input noise enters not only the system dynamics but also the cost functional, and its total effect on the gain eventually vanishes. Consequently, the optimal gain and its associated Riccati recursion need not be recomputed, and the increment in the optimal cost can be readily evaluated in closed form. This result facilitates the use of risk-sensitive LQR in applications that employ input randomization for privacy or exploration, such as watermarking for replay attack detection, differential privacy, and path integral control.

math.OC↗
arXiv2026

Reduced order model for parametric Boltzmann equation and its application to inverse problems

The Boltzmann equation plays an important role in modeling mesoscopic behavior in a wide range of scientific and engineering applications. However, its numerical solution is computationally expensive due to the high dimensionality of the model and the nonlinear nonlocal collision operator, especially for steady-state problems that require iterative solvers. This cost becomes prohibitive for inverse problems, where the induced optimization problem requires repeated forward solves. In this work, we propose a reduced-order model (ROM) for the parametric Boltzmann equation to address this computational challenge. The ROM constructs a low-dimensional approximation space for the parameter-induced solution manifold through a residual-based greedy strategy, and the reduced solution is then obtained via residual minimization over the reduced space, subject to mass conservation. The overall efficiency of the ROM is achieved by exploiting the quadratic structure of the collision operator and a precomputed separable approximation of the collision kernel. The resulting ROM is further applied to a thermally-driven inverse problem for reconstructing collision parameters from the observed macroscopic temperature data. This is accomplished either by directly replacing the PDE constraint with the ROM, leading to a bilevel optimization formulation, or by reformulating the task as a single-level optimization problem through the Karush--Kuhn--Tucker (KKT) conditions. Numerical experiments in both collision-dominated and transport-dominated cases are performed to demonstrate the efficiency and accuracy of the proposed ROM and its effectiveness in inverse problems. In particular, the resulting inverse problem is computationally much more tractable, achieving speedups of several orders of magnitude over that based on the full-order model while maintaining comparable accuracy.

math.NA↗
arXiv2026

Momentum in large-batch training: Polyak enlarges the critical batch size, Nesterov improves data efficiency

We study when and how momentum improves large-batch training in the one-pass regime, using power-law kernel regression as a tractable setting. We first characterize risk stability through the critical learning rate, defined as the largest learning rate for stable training, and obtain $η_{\mathrm{SGD}}^{\mathrm{crit}}\eqsim 1$, $η_{\mathrm{Polyak}}^{\mathrm{crit}}\eqsim \min\{1,B(1-ρ)\}$, and $η_{\mathrm{Nesterov}}^{\mathrm{crit}}\eqsim \min\{1,B^β(1-ρ)\}$, where $B$ is the batch size, $ρ$ is the momentum factor, and $β>1$ is the capacity exponent. Within this admissible region, we derive scaling laws for the full risk dynamics, capturing the progression from an early transient, through power-law decay, to a noise floor. We then minimize the final-step risk over the admissible learning rates and momentum factors under a fixed data budget, yielding a three-regime batch-size phase diagram that reveals how the role of momentum changes with batch size. Notably, Polyak enlarges the critical batch size, the largest batch size preserving the best small-batch data-scaling exponent, thereby enabling greater parallelism without sacrificing data efficiency. In contrast, Nesterov achieves better data efficiency in the large-batch regime because its look-ahead mechanism suppresses noise accumulation. Numerical experiments validate the predicted stability boundaries, risk dynamics, and batch-size phase diagram.

stat.ML↗
arXiv2026

No-Regret Bayesian Optimization with Finite-Library Input-Warped Kernels

Gaussian-process Bayesian optimization (GP-BO) excels at black-box optimization of costly functions, e.g., hyperparameter optimization (HPO) and multi-agent system (MAS) design. Convergence-rate guarantees exist for select methods, notably GP upper confidence bound (GP-UCB), but require a fixed kernel. Critically, the kernel encodes how input proximity affects objective value similarity. When raw coordinates poorly match this geometry - as with log-scaled hyperparameters or localized peaks - input warping can greatly improve sample efficiency, yet known GP-UCB proofs require a fixed kernel. We propose Finite-Library Input-Warped Bayesian Optimization (FLIWBO), which selects warps from a finite library of smooth input maps by any history-dependent rule. It adapts the input geometry to accelerate learning while retaining high-probability convergence guarantees under mild hypotheses, with an explicit $\sqrt(N_\varepsilon)$ library-size cost. Controlled diagnostics show that finite-library warping repairs planted geometry mismatches and identify FLIWBO failure cases. Across four repeated benchmarks - warped synthetic objectives, a confidence-fence trap, and Fashion-MNIST HPO - FLIWBO-UCB beats raw-coordinate GP-UCB under misspecified geometry, escapes traps that defeat even oracle-warp expected improvement, and recovers much of the gain from manual log scaling, while leading the tested methods that admit a matching regret guarantee. A 20-dimensional MAS design study further shows feasibility under costly noisy evaluations. Code for experiments is available: https://github.com/edvin-ketabati/bogp-paper-experiments.

cs.LG↗
arXiv2026

Discrete Gromov-Wasserstein Duality: Algorithms and Isomorphism Testing

The Gromov-Wasserstein (GW) distance provides a principled framework for aligning metric measure (mm) spaces based solely on their intrinsic structure. Its ability to identify isomorphic representations of distributions across spaces renders it valuable for comparing data where equality up to isomorphism occurs naturally such as in graphs or, more generally, distributions on graphs. Recently, a type of dual form for the GW distance between Euclidean distributions with the squared Euclidean or inner product costs was derived, spurring the development of new statistical and algorithmic results for this setting. This work furnishes a novel duality result for GW distances with and without entropic regularization that is applicable to all finitely supported mm spaces. Leveraging this result, we derive the sample complexity of empirical GW distances between finite mm spaces, as well as limit distributions under proper centering and scaling. Furthermore, we propose new algorithms for solving the regularized GW problem which are subject to formal convergence guarantees. These statistical and algorithmic advancements give rise to a principled and efficient framework for testing whether two distributions on the set of graphs with a fixed number of nodes are isomorphic based on samples.

math.ST↗