arXiv subjects

Aurya Javeed

Publications and source records attributed to Aurya Javeed.

8 recordsLinked to original sources

arXiv2026

Duality and Error for Predictively Oriented Inference

Predictively oriented (PrO) inference quantifies uncertainty by selecting a distribution over model parameters to optimize a scoring rule applied to the induced predictive distribution, together with a divergence penalty from a reference distribution. By applying the scoring rule after averaging model densities, PrO inference targets predictive performance, accounting for model misspecification. We focus on the logarithmic score with general $\phi$-divergence regularization. Our contributions are twofold. First, we derive a finite-dimensional dual formulation of PrO inference. For $n$ observations, the dual problem has $n+1$ variables. We establish zero-duality-gap criteria and optimality conditions that relate the primal and dual solutions. When primal and dual solutions exist, these conditions yield a semi-analytical representation of the PrO posterior and certificates for assessing the accuracy of numerical solutions. For Kullback--Leibler regularization, the posterior has an exponential form. Second, we derive a finite-sample excess predictive-risk bound for approximate PrO posteriors that separates sampling fluctuation, approximation under a divergence budget, regularization, and numerical optimization error. The result applies even when the benchmark predictive risk is not attained by any probability distribution over the model parameters having finite divergence from the reference distribution. We use an exactly solvable categorical example to show that predictive-risk convergence can imply convergence to a unique predictive distribution even though the parameter distributions have no weak limit on the original parameter space. The example also shows that different $\phi$-divergences can require different regularization schedules. We conclude with a misspecified Gaussian location-mixture example that illustrates the dual computation, primal recovery, and numerical accuracy checks.

stat.ME↗

arXiv2026

Statistical Inference for Scenario-Based Dynamic Optimization under Uncertainty

Motivated by batch and semi-batch process operation, we study finite-horizon open-loop dynamic optimization problems with uncertain parameters. A common computational approach replaces the expected performance criterion by an average over finitely many sampled parameter realizations. We develop a statistical theory for the resulting sample-based optimal value as an estimator of the population optimal value. The analysis is based on a stability estimate showing that terminal losses depend Lipschitz continuously on the time-integrated control, which records the cumulative input delivered up to each time. This estimate yields a functional central limit theorem for the sample-based objective and a statistical limit theorem for the corresponding optimal value error. As a consequence, we obtain confidence intervals for the population optimal value. When the population optimizer is unique, the limit is Gaussian and leads to a plug-in confidence interval. When multiple optimal policies may exist, we use a subsampling confidence interval that does not require uniqueness. The methodology is illustrated on two fed-batch case studies in which feed-rate profiles are optimized under parametric uncertainty.

math.OC↗

arXiv2025

SUPN: Shallow Universal Polynomial Networks

Deep neural networks (DNNs) and Kolmogorov-Arnold networks (KANs) are popular methods for function approximation due to their flexibility and expressivity. However, they typically require a large number of trainable parameters to produce a suitable approximation. Beyond making the resulting network less transparent, overparameterization creates a large optimization space, likely producing local minima in training that have quite different generalization errors. In this case, network initialization can have an outsize impact on the model's out-of-sample accuracy. For these reasons, we propose shallow universal polynomial networks (SUPNs). These networks replace all but the last hidden layer with a single layer of polynomials with learnable coefficients, leveraging the strengths of DNNs and polynomials to achieve sufficient expressivity with far fewer parameters. We prove that SUPNs converge at the same rate as the best polynomial approximation of the same degree, and we derive explicit formulas for quasi-optimal SUPN parameters. We complement theory with an extensive suite of numerical experiments involving SUPNs, DNNs, KANs, and polynomial projection in one, two, and ten dimensions, consisting of over 13,000 trained models. On the target functions we numerically studied, for a given number of trainable parameters, the approximation error and variability are often lower for SUPNs than for DNNs and KANs by an order of magnitude. In our examples, SUPNs even outperform polynomial projection on non-smooth functions.

cs.LG↗

arXiv2025

ProxSTORM -- A Stochastic Trust-Region Algorithm for Nonsmooth Optimization

We develop a stochastic trust-region algorithm for minimizing the sum of a Lipschitz-smooth but possibly nonconvex function and a convex but possibly nonsmooth function. Such a problem class arises in many applications, including data science, operations research, and PDE-constrained optimization. This algorithm, which we call ProxSTORM, generalizes STORM [15,11]-a stochastic trust-region algorithm for the unconstrained optimization of smooth functions-and the inexact deterministic proximal trust-region algorithm in [5]. In the absence of a nonsmooth term, we recover the original STORM algorithm, moreover, we improve and simplify certain aspects of STORM analysis, while maintaining STORM martingale framework arguments to prove global convergence and an expected complexity bound. We demonstrate ProxSTORM capabilities on neural network training and topology optimization under uncertainty.

math.OC↗

arXiv2024

A Parallel-in-Time Multigrid Preconditioner for Optimal Control

We develop a parallel-in-time multigrid preconditioner for augmented systems. These saddle-point systems are foundational to numerical optimization. Our preconditioner, when paired with a suitable optimization method, accelerates the solution of optimal control problems. We construct the preconditioner by introducing virtual interface variables that enable time-domain decomposition. After permuting the resulting augmented system into block tridiagonal form, we develop a geometric multigrid scheme with a block Jacobi smoother, which parallelizes trivially in time. As the coarse grid solver we use GMRES preconditioned with a symmetric Gauss-Seidel iteration. We use the multigrid scheme to precondition a flexible GMRES [1] iteration for the solution of the augmented system. We combine our preconditioner with the matrix-free sequential quadratic programming (SQP) algorithm [2] to solve optimal control problems involving the van der Pol oscillator and the viscous Burgers' equation. We find that the preconditioner is remarkably effective when the problems are suitably scaled.

math.OC↗

arXiv2022

Machine Learning for CUDA+MPI Design Rules

We present a new strategy for automatically exploring the design space of key CUDA+MPI programs and providing design rules that discriminate slow from fast implementations. In such programs, the order of operations (e.g., GPU kernels, MPI communication) and assignment of operations to resources (e.g., GPU streams) makes the space of possible designs enormous. Systems experts have the task of redesigning and reoptimizing these programs to effectively utilize each new platform. This work provides a prototype tool to reduce that burden. In our approach, a directed acyclic graph of CUDA and MPI operations defines the design space for the program. Monte-Carlo tree search discovers regions of the design space that have large impact on the program's performance. A sequence-to-vector transformation defines features for each explored implementation, and each implementation is assigned a class label according to its relative performance. A decision tree is trained on the features and labels to produce design rules for each class; these rules can be used by systems experts to guide their implementations. We demonstrate our strategy using a key kernel from scientific computing -- sparse-matrix vector multiplication -- on a platform with multiple MPI ranks and GPU streams.

cs.PF↗

arXiv2017

An Uncertainty Principle for Estimates of Floquet Multipliers

We derive a Cramér-Rao lower bound for the variance of Floquet multiplier estimates that have been constructed from stable limit cycles perturbed by noise. To do so, we consider perturbed periodic orbits in the plane. We use a periodic autoregressive process to model the intersections of these orbits with cross sections, then passing to the limit of a continuum of sections to obtain a bound that depends on the continuous flow restricted to the (nontrivial) Floquet mode. We compare our bound against the empirical variance of estimates constructed using several cross sections. The section-based estimates are close to being optimal. We posit that the utility of our bound persists in higher dimensions when computed along Floquet modes for real and distinct multipliers. Our bound elucidates some of the empirical observations noted in the literature; e.g., (a) it is the number of cycles (as opposed to the frequency of observations) that drives the variance of estimates to zero, and (b) the estimator variance has a positive lower bound as the noise amplitude tends to zero.

math.DS↗

arXiv2017

Timing Observations of Diffusions

This paper addresses a problem in experimental design: We consider Itô diffusions specified by some $θ\in \mathbb{R}$ and assume that we are allowed to observe their sample paths only $n$ times before a terminal time $τ< \infty$. We propose a policy for timing these observations to optimally estimate $θ$. Our policy is adaptive (meaning it leverages earlier observations), and it maximizes the expected Fisher information for $θ$ carried by the observations. In numerical studies, this design reduces the variation of estimated parameters by as much as 75% relative to observations spaced uniformly in time. The policy depends on the value of the parameter being estimated, so we also discuss strategies for incorporating Bayesian priors over $θ$.

stat.ME↗