arXiv subjects

Mihailo R. Jovanović

Publications and source records attributed to Mihailo R. Jovanović.

At least 19 recordsLinked to original sources

arXiv2026

From oblique-wave forcing to streak reinforcement: A perturbation-based frequency-response framework

We develop a perturbation-based frequency-response framework for analyzing amplification mechanisms that are central to subcritical routes to transition in wall-bounded shear flows. By systematically expanding the input-output dynamics of fluctuations about the laminar base flow with respect to forcing amplitude, we establish a rigorous correspondence between linear resolvent analysis and higher-order nonlinear interactions. At second order, quadratic interactions of unsteady oblique waves generate steady streamwise streaks via the lift-up mechanism. We demonstrate that the spatial structure of these streaks is captured by the second output singular function of the streamwise-constant resolvent operator. At higher orders, nonlinear coupling between oblique waves and induced streaks acts as structured forcing of the laminar linearized dynamics, yielding additional streak components whose relative phase governs reinforcement or attenuation of the leading-order streak response. Our analysis identifies a critical forcing amplitude marking the breakdown of the weakly nonlinear regime, beyond which direct numerical simulations exhibit sustained unsteadiness. We show that this breakdown coincides with the onset of secondary instability, revealing that the nonlinear interactions responsible for streak formation also drive the modal growth central to classical transition theory. The resulting framework provides a mechanistically transparent and computationally efficient description of transition that unifies non-modal amplification, streak formation, and modal instability within a single formulation derived directly from the Navier-Stokes equations.

physics.flu-dyn↗

arXiv2026

Stability of Primal-Dual Gradient Flow Dynamics for Multi-Block Convex Optimization Problems

We examine stability properties of primal-dual gradient flow dynamics for composite convex optimization problems with multiple, possibly nonsmooth, terms in the objective function under the generalized consensus constraint. The proposed dynamics are based on the proximal augmented Lagrangian and they provide a viable alternative to ADMM which faces significant challenges from both analysis and implementation viewpoints in large-scale multi-block scenarios. In contrast to customized algorithms with individualized convergence guarantees, we develop a systematic approach for solving a broad class of challenging composite optimization problems. We leverage various structural properties to establish global (exponential) convergence guarantees for the proposed dynamics. Our assumptions are much weaker than those required to prove (exponential) stability of primal-dual dynamics as well as (linear) convergence of discrete-time methods such as standard two-block and multi-block ADMM and EXTRA algorithms. Finally, we show necessity of some of our structural assumptions for exponential stability and provide computational experiments to demonstrate the convenience of the proposed approach for parallel and distributed computing applications.

math.OC↗

arXiv2026

Tannenbaum's gain-margin optimization meets Polyak's heavy-ball algorithm

This paper highlights an apparent, yet relatively unknown link between algorithm design in optimization theory and controller synthesis in robust control. Specifically, quadratic optimization can be recast as a regulation problem within the framework of $\mathcal{H}_\infty$ control. From this vantage point, the optimality of Polyak's fastest heavy-ball algorithm can be ascertained as a solution to a gain margin optimization problem. The approach is independent of Polyak's original and brilliant argument, and relies on foundational work by Tannenbaum, who introduced and solved gain margin optimization via Nevanlinna--Pick interpolation theory. The link between first-order optimization methods and robust control sheds new light on the limits of algorithmic performance of such methods, and suggests a framework where similar computational tasks can be systematically studied and algorithms optimized. In particular, it raises the question as to whether periodically scheduled algorithms can achieve faster rates for quadratic optimization, in a manner analogous to periodic control that extends the gain margin beyond that of time-invariant control. This turns out not to be the case, due to the analytic obstruction of a transmission zero that is inherent in causal schemes. Interestingly, this obstruction can be removed with implicit algorithms, cast as feedback regulation problems with causal, but not strictly causal dynamics, thereby devoid of the transmission zero at infinity and able to achieve superior convergence rates.

eess.SY↗

arXiv2025

Convergence and sample complexity of natural policy gradient primal-dual methods for constrained MDPs

We study the sequential decision making problem of maximizing the expected total reward while satisfying a constraint on the expected total utility. We employ the natural policy gradient method to solve the discounted infinite-horizon optimal control problem for Constrained Markov Decision Processes (constrained MDPs). Specifically, we propose a new Natural Policy Gradient Primal-Dual (NPG-PD) method that updates the primal variable via natural policy gradient ascent and the dual variable via projected subgradient descent. Although the underlying maximization involves a nonconcave objective function and a nonconvex constraint set, under the softmax policy parametrization, we prove that our method achieves global convergence with sublinear rates regarding both the optimality gap and the constraint violation. Such convergence is independent of the size of the state-action space, i.e., it is~dimension-free. Furthermore, for log-linear and general smooth policy parametrizations, we establish sublinear convergence rates up to a function approximation error caused by restricted policy parametrization. We also provide convergence and finite-sample complexity guarantees for two sample-based NPG-PD algorithms. We use a set of computational experiments to showcase the effectiveness of our approach.

math.OC↗

arXiv2025

The role of communication delays in the optimal control of spatially invariant systems

We study optimal proportional feedback controllers for spatially invariant systems when the controller has access to delayed state measurements received from different spatial locations. We analyze how delays affect the spatial locality of the optimal feedback gain leveraging the problem decoupling in the spatial frequency domain. For the cases of expensive control and small delay, we provide exact expressions of the optimal controllers in the limit for infinite control weight and vanishing delay, respectively. In the expensive control regime, the optimal feedback control law decomposes into a delay-aware filtering of the delayed state and the optimal controller in the delay-free setting. Under small delays, the optimal controller is a perturbation of the delay-free one which depends linearly on the delay. We illustrate our analytical findings with a reaction-diffusion process over the real line and a multi-agent system coupled through circulant matrices, showing that delays reduce the effectiveness of optimal feedback control and may require each subsystem within a distributed implementation to communicate with farther-away locations.

math.OC↗

arXiv2025

Optimal Network Topology of Multi-Agent Systems subject to Computation and Communication Latency (with proofs)

We study minimum-variance feedback-control design for a networked control system with retarded dynamics, where inter-agent communication is subject to latency. We prove that such a design can be solved efficiently for circular formations and compute near-optimal control gains in closed form. We show that the centralized control architecture is in general suboptimal when the communication increase with the number of links, and propose a control-driven optimization of the network topology.

eess.SY↗

arXiv2024

Accelerated forward-backward and Douglas-Rachford splitting dynamics

We examine convergence properties of continuous-time variants of accelerated Forward-Backward (FB) and Douglas-Rachford (DR) splitting algorithms for nonsmooth composite optimization problems. When the objective function is given by the sum of a quadratic and a nonsmooth term, we establish accelerated sublinear and exponential convergence rates for convex and strongly convex problems, respectively. Moreover, for FB splitting dynamics, we demonstrate that accelerated exponential convergence rate carries over to general strongly convex problems. In our Lyapunov-based analysis we exploit the variable-metric gradient interpretations of FB and DR splittings to obtain smooth Lyapunov functions that allow us to establish accelerated convergence rates. We provide computational experiments to demonstrate the merits and the effectiveness of our analysis.

math.OC↗

arXiv2024

Stability properties of gradient flow dynamics for the symmetric low-rank matrix factorization problem

The symmetric low-rank matrix factorization serves as a building block in many learning tasks, including matrix recovery and training of neural networks. However, despite a flurry of recent research, the dynamics of its training via non-convex factorized gradient-descent-type methods is not fully understood especially in the over-parameterized regime where the fitted rank is higher than the true rank of the target matrix. To overcome this challenge, we characterize equilibrium points of the gradient flow dynamics and examine their local and global stability properties. To facilitate a precise global analysis, we introduce a nonlinear change of variables that brings the dynamics into a cascade connection of three subsystems whose structure is simpler than the structure of the original system. We demonstrate that the Schur complement to a principal eigenspace of the target matrix is governed by an autonomous system that is decoupled from the rest of the dynamics. In the over-parameterized regime, we show that this Schur complement vanishes at an $O(1/t)$ rate, thereby capturing the slow dynamics that arises from excess parameters. We utilize a Lyapunov-based approach to establish exponential convergence of the other two subsystems. By decoupling the fast and slow parts of the dynamics, we offer new insight into the shape of the trajectories associated with local search algorithms and provide a complete characterization of the equilibrium points and their global stability properties. Such an analysis via nonlinear control techniques may prove useful in several related over-parameterized problems.

cs.LG↗

arXiv2024

From exponential to finite/fixed-time stability: Applications to optimization

The development of finite/fixed-time stable optimization algorithms typically involves study of specific problem instances. The lack of a unified framework hinders understanding of more sophisticated algorithms, e.g., primal-dual gradient flow dynamics. The purpose of this paper is to address the following question: Given an exponentially stable optimization algorithm, can it be modified to obtain a finite/fixed-time stable algorithm? We provide an affirmative answer, demonstrate how the solution can be computed on a finite-time interval via a simple scaling of the right-hand-side of the original dynamics, and certify the desired properties of the modified algorithm using the Lyapunov function that proves exponential stability of the original system. Finally, we examine nonsmooth composite optimization problems and smooth problems with linear constraints to demonstrate the merits of our approach.

math.OC↗

arXiv2024

Tradeoffs between convergence rate and noise amplification for momentum-based accelerated optimization algorithms

We study momentum-based first-order optimization algorithms in which the iterations utilize information from the two previous steps and are subject to an additive white noise. This setup uses noise to account for uncertainty in either gradient evaluation or iteration updates, and it includes Polyak's heavy-ball and Nesterov's accelerated methods as special cases. For strongly convex quadratic problems, we use the steady-state variance of the error in the optimization variable to quantify noise amplification and identify fundamental stochastic performance tradeoffs. Our approach utilizes the Jury stability criterion to provide a novel geometric characterization of conditions for linear convergence, and it reveals the relation between the noise amplification and convergence rate as well as their dependence on the condition number and the constant algorithmic parameters. This geometric insight leads to simple alternative proofs of standard convergence results and allows us to establish ``uncertainty principle'' of strongly convex optimization: for the two-step momentum method with linear convergence rate, the lower bound on the product between the settling time and noise amplification scales quadratically with the condition number. Our analysis also identifies a key difference between the gradient and iterate noise models: while the amplification of gradient noise can be made arbitrarily small by sufficiently decelerating the algorithm, the best achievable variance for the iterate noise model increases linearly with the settling time in the decelerating regime. Finally, we introduce two parameterized families of algorithms that strike a balance between noise amplification and settling time while preserving order-wise Pareto optimality for both noise models.

math.OC↗

arXiv2023

Provably Efficient Generalized Lagrangian Policy Optimization for Safe Multi-Agent Reinforcement Learning

We examine online safe multi-agent reinforcement learning using constrained Markov games in which agents compete by maximizing their expected total rewards under a constraint on expected total utilities. Our focus is confined to an episodic two-player zero-sum constrained Markov game with independent transition functions that are unknown to agents, adversarial reward functions, and stochastic utility functions. For such a Markov game, we employ an approach based on the occupancy measure to formulate it as an online constrained saddle-point problem with an explicit constraint. We extend the Lagrange multiplier method in constrained optimization to handle the constraint by creating a generalized Lagrangian with minimax decision primal variables and a dual variable. Next, we develop an upper confidence reinforcement learning algorithm to solve this Lagrangian problem while balancing exploration and exploitation. Our algorithm updates the minimax decision primal variables via online mirror descent and the dual variable via projected gradient step and we prove that it enjoys sublinear rate $ O((|X|+|Y|) L \sqrt{T(|A|+|B|)}))$ for both regret and constraint violation after playing $T$ episodes of the game. Here, $L$ is the horizon of each episode, $(|X|,|A|)$ and $(|Y|,|B|)$ are the state/action space sizes of the min-player and the max-player, respectively. To the best of our knowledge, we provide the first provably efficient online safe reinforcement learning algorithm in constrained Markov games.

cs.LG↗

arXiv2023

Can Decentralized Control Outperform Centralized? The Role of Communication Latency

In this paper, we examine the influence of communication latency on performance of networked control systems. Even though distributed control architectures offer advantages in terms of communication, maintenance costs, and scalability, it is an open question how communication latency that varies with network topology influences closed-loop performance. For networks in which delays increase with the number of links, we establish the existence of a fundamental performance trade-off that arises from control architecture. In particular, we utilize consensus dynamics with single- and double-integrator agents to show that, if delays increase fast enough, a sparse controller with nearest neighbor interactions can outperform the centralized one with all-to-all communication topology.

math.OC↗

arXiv2022

Independent Policy Gradient for Large-Scale Markov Potential Games: Sharper Rates, Function Approximation, and Game-Agnostic Convergence

We examine global non-asymptotic convergence properties of policy gradient methods for multi-agent reinforcement learning (RL) problems in Markov potential games (MPG). To learn a Nash equilibrium of an MPG in which the size of state space and/or the number of players can be very large, we propose new independent policy gradient algorithms that are run by all players in tandem. When there is no uncertainty in the gradient evaluation, we show that our algorithm finds an $ε$-Nash equilibrium with $O(1/ε^2)$ iteration complexity which does not explicitly depend on the state space size. When the exact gradient is not available, we establish $O(1/ε^5)$ sample complexity bound in a potentially infinitely large state space for a sample-based algorithm that utilizes function approximation. Moreover, we identify a class of independent policy gradient algorithms that enjoys convergence for both zero-sum Markov games and Markov cooperative games with the players that are oblivious to the types of games being played. Finally, we provide computational experiments to corroborate the merits and the effectiveness of our theoretical developments.

cs.LG↗

arXiv2022

Oblique transition in hypersonic double-wedge flow

We utilize resolvent and weakly nonlinear analyses in combination with direct numerical simulations (DNS) to identify mechanisms for oblique transition in a Mach 5 hypersonic flow over an adiabatic slender double-wedge. Even though the laminar separated flow is globally stable, resolvent analysis demonstrates significant amplification of unsteady external disturbances to the linearized flow equations. These disturbances are introduced upstream of the separation zone and they lead to the appearance of oblique waves further downstream. We demonstrate that large amplification of oblique waves arises from the growth of fluctuation shear stress due to streamline curvature of the laminar base flow in the separated shear layer. This is in contrast to the attached boundary layers, where no such mechanism exists. We also use a weakly nonlinear analysis to show that the resolvent operator associated with linearization around the laminar base flow governs the evolution of steady reattachment streaks that arise from quadratic interactions of unsteady oblique waves. These quadratic interactions generate vortical excitations in the reattaching shear layer which lead to the formation of streaks in the recirculation zone and their subsequent amplification, breakdown, and transition to turbulence downstream. Our analysis of the energy budget shows that deceleration of the base flow near reattachment is primarily responsible for amplification of steady streaks. Finally, we employ DNS to examine latter stages of transition to turbulence and demonstrate the predictive power of a weakly nonlinear input-output framework in uncovering triggering mechanisms for oblique transition in separated high-speed boundary layer flows.

physics.flu-dyn↗

arXiv2021

Transient growth of accelerated optimization algorithms

Optimization algorithms are increasingly being used in applications with limited time budgets. In many real-time and embedded scenarios, only a few iterations can be performed and traditional convergence metrics cannot be used to evaluate performance in these non-asymptotic regimes. In this paper, we examine the transient behavior of accelerated first-order optimization algorithms. For convex quadratic problems, we employ tools from linear systems theory to show that transient growth arises from the presence of non-normal dynamics. We identify the existence of modes that yield an algebraic growth in early iterations and quantify the transient excursion from the optimal solution caused by these modes. For strongly convex smooth optimization problems, we utilize the theory of integral quadratic constraints (IQCs) to establish an upper bound on the magnitude of the transient response of Nesterov's accelerated algorithm. We show that both the Euclidean distance between the optimization variable and the global minimizer and the rise time to the transient peak are proportional to the square root of the condition number of the problem. Finally, for problems with large condition numbers, we demonstrate tightness of the bounds that we derive up to constant factors.

math.OC↗

arXiv2021

Fast Multi-Agent Temporal-Difference Learning via Homotopy Stochastic Primal-Dual Optimization

We study the policy evaluation problem in multi-agent reinforcement learning where a group of agents, with jointly observed states and private local actions and rewards, collaborate to learn the value function of a given policy via local computation and communication over a connected undirected network. This problem arises in various large-scale multi-agent systems, including power grids, intelligent transportation systems, wireless sensor networks, and multi-agent robotics. When the dimension of state-action space is large, the temporal-difference learning with linear function approximation is widely used. In this paper, we develop a new distributed temporal-difference learning algorithm and quantify its finite-time performance. Our algorithm combines a distributed stochastic primal-dual method with a homotopy-based approach to adaptively adjust the learning rate in order to minimize the mean-square projected Bellman error by taking fresh online samples from a causal on-policy trajectory. We explicitly take into account the Markovian nature of sampling and improve the best-known finite-time error bound from $O(1/\sqrt{T})$ to~$O(1/T)$, where $T$ is the total number of iterations.

math.OC↗

arXiv2021

Convergence and sample complexity of gradient methods for the model-free linear quadratic regulator problem

Model-free reinforcement learning attempts to find an optimal control action for an unknown dynamical system by directly searching over the parameter space of controllers. The convergence behavior and statistical properties of these approaches are often poorly understood because of the nonconvex nature of the underlying optimization problems and the lack of exact gradient computation. In this paper, we take a step towards demystifying the performance and efficiency of such methods by focusing on the standard infinite-horizon linear quadratic regulator problem for continuous-time systems with unknown state-space parameters. We establish exponential stability for the ordinary differential equation (ODE) that governs the gradient-flow dynamics over the set of stabilizing feedback gains and show that a similar result holds for the gradient descent method that arises from the forward Euler discretization of the corresponding ODE. We also provide theoretical bounds on the convergence rate and sample complexity of the random search method with two-point gradient estimates. We prove that the required simulation time for achieving $ε$-accuracy in the model-free setup and the total number of function evaluations both scale as $\log \, (1/ε)$.

math.OC↗

arXiv2021

Well-conditioned ultraspherical and spectral integration methods for resolvent analysis of channel flows of Newtonian and viscoelastic fluids

Modal and nonmodal analyses of fluid flows provide fundamental insight into the early stages of transition to turbulence. Eigenvalues of the dynamical generator govern temporal growth or decay of individual modes, while singular values of the frequency response operator quantify the amplification of disturbances for linearly stable flows. In this paper, we develop well-conditioned ultraspherical and spectral integration methods for frequency response analysis of channel flows of Newtonian and viscoelastic fluids. Even if a discretization method is well-conditioned, we demonstrate that calculations can be erroneous if singular values are computed as the eigenvalues of a cascade connection of the frequency response operator and its adjoint. To address this issue, we utilize a feedback interconnection of the frequency response operator with its adjoint to avoid computation of inverses and facilitate robust singular value decomposition. Specifically, in contrast to conventional spectral collocation methods, the proposed method (i) produces reliable results in channel flows of viscoelastic fluids at high Weissenberg numbers ($\sim 500$); and (ii) does not require a staggered grid for the equations in primitive variables.

physics.flu-dyn↗