arXiv subjects

Liviu Aolaritei

Publications and source records attributed to Liviu Aolaritei.

At least 19 recordsLinked to original sources

arXiv2026

Beyond Optimal Rates in Stochastic Optimization: Trajectory-Adaptive Stopping Rules

Stochastic gradient descent (SGD) is typically analyzed at a deterministic horizon chosen before the algorithm is run, even though practical stopping decisions are made adaptively by inspecting the evolving trajectory. This mismatch creates a fundamental certification problem: fixed-time guarantees do not generally remain valid at data-dependent stopping times, while deterministic horizons derived from worst-case bounds can be highly conservative. We address this problem for strongly convex stochastic optimization by constructing fully observable, trajectory-adaptive upper confidence sequences for the squared distance of the last iterate to the optimizer and the suboptimality of a weighted average. These bounds hold simultaneously over time, attain the optimal $1/t$ decay rate up to iterated-logarithmic factors in the worst case, and adapt to the realized stochastic gradients, allowing SGD to stop as soon as a prescribed accuracy is certified without sacrificing statistical validity. Our approach treats the evolving SGD trajectory as a sequential experiment whose observations provide evidence about the unknown optimization error. To formalize this perspective, we develop new recursive confidence-sequence techniques and a general time-uniform empirical Bernstein inequality for adapted processes with time-varying conditional means and predictable ranges that may grow without bound. We further extend these confidence-sequence constructions to minibatch SGD, with the empirical Bernstein bounds exploiting the realized second-moment structure within each minibatch. Numerical experiments show that the resulting stopping rules can require several orders of magnitude fewer iterations than natural deterministic horizons.

cs.LG↗

arXiv2026

Diffusion-Robust Optimization over Graphs

We introduce a diffusion-based uncertainty model for robust optimization on directed graphs, in which perturbations of edge weights propagate along adjacent edges and satisfy conservation constraints at nodes. This topology-aware structure is natural in networked systems where uncertainty is induced by flows and local interactions, including transportation, logistics, communication, and energy networks. We analyze how such diffusive uncertainty reshapes the computational landscape of robust graph optimization. We focus on two canonical combinatorial graph problems, shortest path and the traveling salesman problem (TSP), which provide complementary benchmarks: shortest path is polynomial-time solvable in the nominal setting, whereas TSP is already NP-hard. We show that, for shortest path, propagation depth induces a sharp transition between tractable and intractable robust counterparts. For the traveling salesman problem, robustness often adds no computational complexity beyond ordinary TSP, because the structure of Hamiltonian cycles makes the fixed-tour adversarial problem collapse to explicit formulas. Together, these results show that topology-aware uncertainty can fundamentally change robust combinatorial optimization, with tractability governed by the interaction between propagation, budget geometry, and the structure of feasible solutions.

math.OC↗

arXiv2026

Minimum Volume Conformal Sets for Multivariate Regression

Conformal prediction provides a principled framework for constructing predictive sets with finite-sample validity. While much of the focus has been on univariate response variables, existing multivariate methods either impose rigid geometric assumptions or rely on flexible but computationally expensive approaches that do not explicitly optimize prediction set volume. We propose an optimization-driven framework based on a novel loss function that directly learns minimum-volume covering sets while ensuring valid coverage. This formulation naturally induces a new nonconformity score for conformal prediction, which adapts to the residual distribution and covariates. Our approach optimizes over prediction sets defined by arbitrary norm balls, including single and multi-norm formulations. Additionally, by jointly optimizing both the predictive model and predictive uncertainty, we obtain prediction sets that are tight, informative, and computationally efficient, as demonstrated in our experiments on real-world datasets.

stat.ML↗

arXiv2026

Stopping Rules for Stochastic Gradient Descent via Anytime-Valid Confidence Sequences

The problem of stopping stochastic gradient descent (SGD) in an online manner, based solely on the observed trajectory, is a challenging theoretical problem with significant consequences for applications. While SGD is routinely monitored as it runs, the classical theory of SGD provides guarantees only at pre-specified iteration horizons and offers no valid way to decide, based on the observed trajectory, when further computation is justified. We address this longstanding gap by developing anytime-valid confidence sequences for stochastic gradient methods, which remain valid under continuous monitoring and directly induce statistically valid, trajectory-dependent stopping rules: stop as soon as the current upper confidence bound on an appropriate performance measure falls below a user-specified tolerance. The confidence sequences are constructed using nonnegative supermartingales, are time-uniform, and depend only on observable quantities along the SGD trajectory, without requiring prior knowledge of the optimization horizon. In convex optimization, this yields anytime-valid certificates for weighted suboptimality of projected SGD under general stepsize schedules, without assuming smoothness or strong convexity. In nonconvex optimization, it yields time-uniform certificates for weighted first-order stationarity under smoothness assumptions. We further characterize the stopping-time complexity of the resulting stopping rules under standard stepsize schedules. To the best of our knowledge, this is the first framework that provides statistically valid, time-uniform stopping rules for SGD across both convex and nonconvex settings based solely on its observed trajectory.

math.OC↗

arXiv2026

Stochastic Optimization with Optimal Importance Sampling

Importance Sampling (IS) is a widely used variance reduction technique for enhancing the efficiency of Monte Carlo methods, particularly in rare-event simulation and related applications. Despite its effectiveness, the performance of IS is highly sensitive to the choice of the proposal distribution and often requires stochastic calibration. While the design and analysis of IS have been extensively studied in estimation settings, applying IS within stochastic optimization introduces a lesser-known fundamental challenge: the decision variable and the importance sampling distribution are mutually dependent, creating a circular optimization structure. This interdependence complicates both convergence analysis and variance control. In this paper, we consider the generic setting of convex stochastic optimization with linear constraints. We propose a single-loop stochastic approximation algorithm, based on a variant of Nesterov's dual averaging, that jointly updates the decision variable and the importance sampling distribution, notably without time-scale separation or nested optimization. The method is globally convergent and achieves the minimal asymptotic variance among stochastic gradient schemes, which moreover matches the performance of an oracle sampler adapted to the optimal solution and thus effectively resolves the circular optimization challenge.

math.OC↗

arXiv2025

Hedging against Black Swans in Day-Ahead Energy Markets

Renewable generators must commit to day-ahead market bids despite uncertainty in both production and real-time prices. While forecasts provide valuable guidance, rare and unpredictable extreme events (so-called black swans) can cause substantial financial losses. This paper models the nomination problem as an instance of optimal transport-based distributionally robust optimization (OT-DRO), a principled framework that balances risk and performance by accounting not only for the severity of deviations but also for their likelihood. The resulting formulation yields a tractable, data-driven strategy that remains competitive under normal conditions while providing effective protection against extreme price spikes. Using four years of Finnish wind farm and market data, we demonstrate that OT-DRO consistently outperforms forecast-based nominations and significantly mitigates losses during black swan events.

math.OC↗

arXiv2025

Nash Equilibria, Regularization and Computation in Optimal Transport-Based Distributionally Robust Optimization

We study optimal transport-based distributionally robust optimization problems where a fictitious adversary, often envisioned as nature, can choose the distribution of the uncertain problem parameters by reshaping a prescribed reference distribution at a finite transportation cost. In this framework, we show that robustification is intimately related to various forms of variation and Lipschitz regularization even if the transportation cost function fails to be (some power of) a metric. We also derive conditions for the existence and the computability of a Nash equilibrium between the decision-maker and nature, and we demonstrate numerically that nature's Nash strategy can be viewed as a distribution that is supported on remarkably deceptive adversarial samples. Finally, we identify practically relevant classes of optimal transport-based distributionally robust optimization problems that can be addressed with efficient gradient descent algorithms even if the loss function or the transportation cost function are nonconvex (but not both at the same time).

math.OC↗

arXiv2025

Learn to Bid as a Price-Maker Wind Power Producer

Wind power producers (WPPs) participating in short-term power markets face significant imbalance costs due to their non-dispatchable and variable production. While some WPPs have a large enough market share to influence prices with their bidding decisions, existing optimal bidding methods rarely account for this aspect. Price-maker approaches typically model bidding as a bilevel optimization problem, but these methods require complex market models, estimating other participants' actions, and are computationally demanding. To address these challenges, we propose an online learning algorithm that leverages contextual information to optimize WPP bids in the price-maker setting. We formulate the strategic bidding problem as a contextual multi-armed bandit, ensuring provable regret minimization. The algorithm's performance is evaluated against various benchmark strategies using a numerical simulation of the German day-ahead and real-time markets.

cs.LG↗

arXiv2025

Revisiting mean estimation over $\ell_p$ balls: Is the MLE optimal?

We revisit the problem of mean estimation in the Gaussian sequence model with $\ell_p$ constraints for $p \in [0, \infty]$. We demonstrate two phenomena for the behavior of the maximum likelihood estimator (MLE), which depend on the noise level, the radius of the (quasi)norm constraint, the dimension, and the norm index $p$. First, if $p$ lies between $0$ and $1 + Θ(\tfrac{1}{\log d})$, inclusive, or if it is greater than or equal to $2$, the MLE is minimax rate-optimal for all noise levels and all constraint radii. On the other hand, for the remaining norm indices -- namely, if $p$ lies between $1 + Θ(\tfrac{1}{\log d})$ and $2$ -- here is a more striking behavior: the MLE is minimax rate-suboptimal, despite its nonlinearity in the observations, for essentially all noise levels and constraint radii for which nonlinear estimates are necessary for minimax-optimal estimation. Our results imply that when given $n$ independent and identically distributed Gaussian samples, the MLE can be suboptimal by a polynomial factor in the sample size. Our lower bounds are constructive: whenever the MLE is rate-suboptimal, we provide explicit instances on which the MLE provably incurs suboptimal risk. Finally, in the non-convex case -- namely when $p < 1$ -- we develop sharp local Gaussian width bounds, which may be of independent interest.

math.ST↗

arXiv2025

Valid Selection among Conformal Sets

Conformal prediction offers a distribution-free framework for constructing prediction sets with coverage guarantees. In practice, multiple valid conformal prediction sets may be available, arising from different models or methodologies. However, selecting the most desirable set, such as the smallest, can invalidate the coverage guarantees. To address this challenge, we propose a stability-based approach that ensures coverage for the selected prediction set. We extend our results to the online conformal setting, propose several refinements in settings where additional structure is available, and demonstrate its effectiveness through experiments.

stat.ML↗

arXiv2025

Conformal Prediction under Levy-Prokhorov Distribution Shifts: Robustness to Local and Global Perturbations

Conformal prediction provides a powerful framework for constructing prediction intervals with finite-sample guarantees, yet its robustness under distribution shifts remains a significant challenge. This paper addresses this limitation by modeling distribution shifts using Levy-Prokhorov (LP) ambiguity sets, which capture both local and global perturbations. We provide a self-contained overview of LP ambiguity sets and their connections to popular metrics such as Wasserstein and Total Variation. We show that the link between conformal prediction and LP ambiguity sets is a natural one: by propagating the LP ambiguity set through the scoring function, we reduce complex high-dimensional distribution shifts to manageable one-dimensional distribution shifts, enabling exact quantification of worst-case quantiles and coverage. Building on this analysis, we construct robust conformal prediction intervals that remain valid under distribution shifts, explicitly linking LP parameters to interval width and confidence levels. Experimental results on real-world datasets demonstrate the effectiveness of the proposed approach.

stat.ML↗

arXiv2025

Wasserstein Distributionally Robust Estimation in High Dimensions: Performance Analysis and Optimal Hyperparameter Tuning

Distributionally robust optimization (DRO) has become a powerful framework for estimation under uncertainty, offering strong out-of-sample performance and principled regularization. In this paper, we propose a DRO-based method for linear regression and address a central question: how to optimally choose the robustness radius, which controls the trade-off between robustness and accuracy. Focusing on high-dimensional settings where the dimension and the number of samples are both large and comparable in size, we employ tools from high-dimensional asymptotic statistics to precisely characterize the estimation error of the resulting estimator. Remarkably, this error can be recovered by solving a simple convex-concave optimization problem involving only four scalar variables. This characterization enables efficient selection of the radius that minimizes the estimation error. In doing so, it achieves the same effect as cross-validation, but at a fraction of the computational cost. Numerical experiments confirm that our theoretical predictions closely match empirical performance and that the optimal radius selected through our method aligns with that chosen by cross-validation, highlighting both the accuracy and the practical benefits of our approach.

stat.ML↗

arXiv2023

Distributional Uncertainty Propagation via Optimal Transport

This paper addresses the limitations of standard uncertainty models, e.g., robust (norm-bounded) and stochastic (one fixed distribution, e.g., Gaussian), and proposes to model uncertainty via Optimal Transport (OT) ambiguity sets. These constitute a very rich uncertainty model, which enjoys many desirable geometrical, statistical, and computational properties, and which: (1) naturally generalizes both robust and stochastic models, and (2) captures many additional real-world uncertainty phenomena (e.g., black swan events). Our contributions show that OT ambiguity sets are also analytically tractable: they propagate easily and intuitively through linear and nonlinear (possibly corrupted by noise) transformations, and the result of the propagation is again an OT ambiguity set or can be tightly upper bounded by an OT ambiguity set. In the context of dynamical systems, our results allow us to consider multiple sources of uncertainty (e.g., initial condition, additive noise, multiplicative noise) and to capture in closed-form, via an OT ambiguity set, the resulting uncertainty in the state at any future time. Our results are actionable, interpretable, and readily employable in a great variety of computationally tractable control and estimation formulations. To highlight this, we study three applications in trajectory planning, consensus algorithms, and least squares estimation. We conclude the paper with a list of exciting open problems enabled by our results.

math.OC↗

arXiv2023

Wasserstein Tube MPC with Exact Uncertainty Propagation

We study model predictive control (MPC) problems for stochastic LTI systems, where the noise distribution is unknown, compactly supported, and only observable through a limited number of i.i.d. noise samples. Building upon recent results in the literature, which show that distributional uncertainty can be efficiently captured within a Wasserstein ambiguity set, and that such ambiguity sets propagate exactly through the system dynamics, we start by formulating a novel Wasserstein Tube MPC (WT-MPC) problem, with distributionally robust CVaR constraints. We then show that the WT-MPC problem: (1) is a direct generalization of the (deterministic) Robust Tube MPC (RT-MPC) to the stochastic setting; (2) through a scalar parameter, it interpolates between the data-driven formulation based on sample average approximation and the RT-MPC formulation, allowing us to optimally trade between safety and performance; (3) admits a tractable convex reformulation; and (4) is recursively feasible. We conclude the paper with a numerical comparison of WT-MPC and RT-MPC.

math.OC↗

arXiv2023

Capture, Propagate, and Control Distributional Uncertainty

We study stochastic dynamical systems in settings where only partial statistical information about the noise is available, e.g., in the form of a limited number of noise realizations. Such systems are particularly challenging to analyze and control, primarily due to an absence of a distributional uncertainty model which: (1) is expressive enough to capture practically relevant scenarios; (2) can be easily propagated through system maps; (3) is invariant under propagation; and (4) allows for computationally tractable control actions. In this paper, we propose to model distributional uncertainty via Optimal Transport ambiguity sets and show that such modeling choice satisfies all of the above requirements. We then specialize our results to stochastic LTI systems, and start by showing that the distributional uncertainty can be efficiently captured, with high probability, within an Optimal Transport ambiguity set on the space of noise trajectories. Then, we show that such ambiguity sets propagate exactly through the system dynamics, giving rise to stochastic tubes that contain, with high probability, all trajectories of the stochastic system. Finally, we show that the control task is very interpretable, unveiling an interesting decomposition between the roles of the feedforward and the feedback control terms. Our results are actionable and successfully applied in stochastic reachability analysis and in trajectory planning under distributional uncertainty.

math.OC↗

arXiv2019

Hierarchical and Distributed Monitoring of Voltage Stability in Distribution Networks

We consider the problem of quantifying and assessing the steady-state voltage stability in radial distribution networks. Our approach to the voltage stability problem is based on a local, approximate, and yet highly accurate characterization of the determinant of the Jacobian of the power flow equations parameterized according to the branch-flow model. The proposed determinant approximation allows us to construct a voltage stability index that can be computed in a fully distributed or in a hierarchical fashion, resulting in a scalable approach to the assessment of steady-state voltage stability. Finally, we provide upper bounds for the approximation error and we numerically validate the quality and the robustness of the proposed approximation with the IEEE 123-bus test feeder.

math.OC↗

arXiv2018

Robustness against Disturbances in Power Systems under Frequency Constraints

The wide deployment of renewable generation and the gradual decrease in the overall system inertia make modern power grids more vulnerable to transient instabilities and unacceptable frequency fluctuations. Time-domain simulation-based assessment of the system robustness against uncertain and stochastic disturbances is extremely time-consuming. In this paper, we develop an alternative approach, which has its roots in the input-output stability analysis for Lur'e systems. Our approach consists of a mathematically rigorous characterization of the external disturbances that the power system is transiently stable and the frequency constraints are not violated. The derived certificate is efficiently constructed via convex optimization and is shown to be non-conservative for different IEEE test cases.

math.OC↗

arXiv2018

A Robustness Measure of Transient Stability under Operational Constraints in Power Systems

The aggressive integration of distributed renewable sources is changing the dynamics of the electric power grid in an unexpected manner. As a result, maintaining conventional performance specifications, such as transient stability, may not be sufficient to ensure its reliable operation in stressed conditions. In this paper, we introduce a novel criteria in transient stability with consideration of operational constraints over frequency deviation and angular separation. In addition, we provide a robustness measure of the region of attraction, which can quantify the ability of the post-fault system to remain synchronized even under disturbances. To assess this new stability specification, we adopt the notion of Input-to-State Stability (ISS) to the context of power systems and introduce a new class of convex Lyapunov functions, which will result in tractable convex-optimization-based stability certificates. As a result, we are able to quantify the level of disturbance a power system can withstand while maintaining its safe operation. We illustrate the introduced stability specification and certificate on the IEEE 9 bus system.

math.OC↗