arXiv subjects

Vijay Gupta

Publications and source records attributed to Vijay Gupta.

At least 19 recordsLinked to original sources

arXiv2026

Learning to Control Coupled-Dynamics Environments with Joint Markov Decision Processes

Coupled-dynamics environments expose the one-step outcomes that would follow from several possible counterfactual actions under a common realization of exogenous randomness. The ordinary Markov decision process formalism allows one to reason about the marginal law of each action but discards dependence across these counterfactual outcomes. The Joint Markov decision process (JMDP) formalism preserves that dependence. Prior work established the formalism and solved the fixed-policy joint moment evaluation problem in JMDPs. This paper develops optimal-control methods. We define a nonparametric distributional Bellman optimality operator for JMDPs, and prove that when the induced marginal MDP has a unique optimal policy, its iterates converge in Wasserstein distance to the optimal joint return law. For the first two moments, we establish convergence under a weaker condition that permits several mean-optimal actions as long as their tie resolutions share a second-moment fixed point. We also derive sampled targets for neural approximation.

cs.LG↗

arXiv2026

Stabilization Limits of Payoff-Based Higher-Order Replicator Dynamics

Replicator dynamics (RD) is a fundamental model in learning in games, connecting evolutionary game theory and online learning. This paper studies payoff-based higher-order variants of RD represented as a cascade interconnection between an integrator in parallel with an auxiliary linear time-invariant (LTI) system and the softmax mapping. We investigate learnability of Nash equilibria under this Nash-stationary learning rule. First, we revisit recent results that establish convergence to Nash Equilibrium whenever the auxiliary LTI system is strictly passive and prove a converse passivity result: if the auxiliary LTI system is not passive, then there exists a static strictly contractive game whose interior Nash equilibrium is unstable under the closed-loop learning dynamics. Second, we show that there exists a class of games with isolated interior Nash equilibria that cannot be locally asymptotically stabilized by any payoff-based higher-order RD whose auxiliary LTI system is asymptotically stable and strictly proper. Finally, we show that if Nash stationarity (i.e., all Nash equilibria are stationary points of the learning dynamics) is relaxed, then generalized exponential RD (Ex-RD) can locally asymptotically stabilize a logit equilibrium for any continuously differentiable game. The stabilized equilibrium can be viewed as an entropy-regularized approximate Nash equilibrium.

eess.SY↗

arXiv2026

Comparison of Non-Deterministic Nonlinear Systems

We characterize a notion of system comparison, termed as $(T_e,\gamma,\delta)$-similarity, for non-deterministic nonlinear systems. Building on a similar notion recently proposed for stable linear systems, the proposed notion characterizes the dissimilarity between the outputs, measured using the $L_2$ norm, of two nonlinear dynamical systems in terms of their inputs and disturbances. By establishing a relationship between $(T_e,\gamma,\delta)$-similarity and differential dissipativity, we establish equivalence between $(T_e,\gamma,\delta)$-similarity of nonlinear systems and the $(T_e,\gamma,\delta)$-similarity of their differential dynamics. We characterize the $(T_e,\gamma,\delta)$-similarity for nonlinear systems as a Linear Matrix Inequality feasibility problem and also provide necessary and sufficient conditions for solving this feasibility problem. We demonstrate the utility of the proposed notion through its use in two applications: (i) robust hierarchical control applied to a planar aircraft and (ii) the improvement (or design) of abstract models applied to the Moore-Greitzer model and an electronic circuit.

eess.SY↗

arXiv2026

Active Sensing and Deferred-Decision Trajectory Optimization for Robust Target Identification

We study trajectory optimization in mobile sensing systems that must identify which member of a finite candidate set is the true target, while maintaining reachability to all potential candidate targets, under resource constraints. Deferred-Decision Trajectory Optimization (DDTO) addresses this setting by computing trajectories that reach individual targets but remain coincident for as long as possible before separating toward different targets. We propose Active-Sensing DDTO (AS-DDTO), which extends DDTO by adding a trajectory-dependent information-acquisition term to the planning objective. The resulting planner maintains reachability to candidate targets while biasing the coincident portion of the trajectories toward regions that enable earlier target identification. The framework supports Bayesian updates and conformal candidate-set updates for distance-dependent sensing. We derive a mixed-integer conic reformulation and provide guarantees on recursive feasibility, belief concentration, and fixed-time coverage for the raw conformal candidate set. Numerical simulations show improved target identification compared with standard DDTO under distance-dependent sensing uncertainty and limited sensing budget.

eess.SY↗

arXiv2026

Quotient-Categorical Representations for Bellman-Compatible Average-Reward Distributional Reinforcement Learning

Average-reward reinforcement learning requires estimating the gain and the bias, which is defined only up to an additive constant. This makes direct distributional analogues ill-posed on the real line. We introduce a quotient-space formulation in which state-indexed bias laws are identified up to a common translation, together with a categorical parameterization that respects this symmetry. On this quotient-categorical space, we define a projected average-reward distributional operator and show that it is well-defined, non-expansive in a coordinate Cram\'er metric, and admits fixed points. We then study sampled recursions whose mean-field maps are asynchronous relaxations of this operator. In an idealized centered-reward setting, a one-state temporal-difference update enjoys almost sure convergence together with finite-iteration residual bounds under both i.i.d. and Markovian sampling. When the gain is unknown, we augment the recursion with an online gain estimator, and prove non-expansiveness and Markovian convergence of the resulting coupled scheme. Finally, we show that synchronous exact updates are gain-independent at the quotient-law level, isolating a structural contrast between ideal quotient distributions and practical fixed-grid categorical representations.

cs.LG↗

arXiv2026

The Variational Approach in Filtering and Correlated Noise

The variational formulation of nonlinear filtering due to Mitter and Newton characterizes the filtering distribution as the unique minimizer of a free energy functional involving the relative entropy with respect to the prior and an expected energy. This formulation rests on an absolute continuity condition between the joint path measure and a product reference measure. We prove that this condition necessarily fails whenever the signal and observation diffusions share a common noise source. Specifically we show that the joint and product measures are mutually singular, so no choice of reference measure can salvage the formulation. We then introduce a conditional variational principle that replaces the prior with a reference measure that preserves the noise correlation structure. This generalization recovers the Mitter--Newton formulation as a special case when the noises are independent, and yields an explicit free energy characterization of the filter in the linear correlated-noise setting.

math.PR↗

arXiv2026

Convergence of Payoff-Based Higher-Order Replicator Dynamics in Contractive Games

We study the convergence properties of a payoff-based higher-order version of replicator dynamics, a widely studied model in evolutionary dynamics and game-theoretic learning, in contractive games. Recent work has introduced a control-theoretic perspective for analyzing the convergence of learning dynamics through passivity theory, leading to a classification of learning dynamics based on the passivity notion they satisfy, such as \textdelta-passivity, equilibrium-independent passivity, and incremental passivity. We leverage this framework for the study of higher-order replicator dynamics for contractive games, which form the complement of passive learning dynamics. Standard replicator dynamics can be represented as a cascade interconnection between an integrator and the softmax mapping. Payoff-based higher-order replicator dynamics include a linear time-invariant (LTI) system in parallel with the existing integrator. First, we show that if this added system is strictly passive and asymptotically stable, then the resulting learning dynamics converge locally to the Nash equilibrium in contractive games. Second, we establish global convergence properties using incremental stability analysis for the special case of symmetric matrix contractive games.

eess.SY↗

arXiv2026

Coverage Guarantees for Pseudo-Calibrated Conformal Prediction under Distribution Shift

Conformal prediction (CP) offers distribution-free marginal coverage guarantees under an exchangeability assumption, but these guarantees can fail if the data distribution shifts. We analyze the use of pseudo-calibration as a tool to counter this performance loss under a bounded label-conditional covariate shift model. Using tools from domain adaptation, we derive a lower bound on target coverage in terms of the source-domain loss of the classifier and a Wasserstein measure of the shift. Using this result, we provide a method to design pseudo-calibrated sets that inflate the conformal threshold by a slack parameter to keep target coverage above a prescribed level. Finally, we propose a source-tuned pseudo-calibration algorithm that interpolates between hard pseudo-labels and randomized labels as a function of classifier uncertainty. Numerical experiments show that our bounds qualitatively track pseudo-calibration behavior and that the source-tuned scheme mitigates coverage degradation under distribution shift while maintaining nontrivial prediction set sizes.

cs.LG↗

arXiv2026

Robust Filtering of L\'evy-driven Stochastic Models

We study robust nonlinear filtering for stochastic models driven by L\'evy processes, where the signal and observation processes are coupled through common Brownian and jump noise. Robustness, defined as the continuous dependence of the filter on the observation path, is essential whenever the observation process deviates from the idealized model, for instance when a path must be reconstructed from discrete-time samples. This question is well understood for continuous semimartingale systems but largely open in the presence of jumps. We construct a version of the filter and establish its continuity in two regimes. For processes with finitely many jumps on compact intervals, we prove continuity in both the rough $p$-variation and $p$-variation topologies on cadlag path space, without requiring a separability condition on the jump coefficients. For processes with infinitely many jumps, we prove continuity in a modified rough $p$-variation topology adapted to cadlag geometric rough paths, under an additional separability assumption. In both cases, our approach relies on Stratonovich and Marcus flow decompositions rather than the It\^o-based methods of recent work. The resulting geometric rough-path lifts yield pathwise convergence guarantees and can be constructed directly from discrete observations without knowledge of the underlying probability law.

math.PR↗

arXiv2026

A Survey of Security Challenges and Solutions for Advanced Air Mobility and eVTOL Aircraft

This survey reviews the existing and envisioned security vulnerabilities and defense mechanisms relevant to Advanced Air Mobility (AAM) systems, with a focus on electric vertical takeoff and landing (eVTOL) aircraft. Drawing from vulnerabilities in the avionics in commercial aviation and the automated unmanned aerial systems (UAS), the paper presents a taxonomy of attacks, analyzes mitigation strategies, and proposes a secure system architecture tailored to the future AAM ecosystem. The paper also highlights key threat vectors, including Global Positioning System (GPS) jamming/spoofing, ATC radio frequency misuse, attacks on TCAS and ADS-B, possible backdoor via Electronic Flight Bag (EFB), new vulnerabilities introduced by aircraft automation and connectivity, and risks from flight management system (FMS) software, database and cloud services. Finally, this paper describes emerging defense techniques against these attacks, and open technical problems to address toward better defense mechanisms.

cs.CR↗

arXiv2026

A Survey of Security Challenges and Solutions for UAS Traffic Management (UTM) and small Unmanned Aerial Systems (sUAS)

The rapid growth of small Unmanned Aerial Systems (sUAS) for civil and commercial missions has intensified concerns about their resilience to cyber-security threats. Operating within the emerging UAS Traffic Management (UTM) framework, these lightweight and highly networked platforms depend on secure communication, navigation, and surveillance (CNS) subsystems that are vulnerable to spoofing, jamming, hijacking, and data manipulation. While prior reviews of UAS security addressed these challenges at a conceptual level, a detailed, system-oriented analysis for resource-constrained sUAS remains lacking. This paper presents a comprehensive survey of cyber-security vulnerabilities and defenses tailored to the sUAS and UTM ecosystem. We organize existing research across the full cyber-physical stack, encompassing CNS, data links, sensing and perception, UTM cloud access, and software integrity layers, and classify attack vectors according to their technical targets and operational impacts. Correspondingly, we review defense mechanisms ranging from classical encryption and authentication to adaptive intrusion detection, lightweight cryptography, and secure firmware management. By mapping threats to mitigation strategies and evaluating their scalability and practical effectiveness, this work establishes a unified taxonomy and identifies open challenges for achieving safe, secure, and scalable sUAS operations within future UTM environments.

cs.CR↗

arXiv2025

Linear Quadratic Control with Non-Markovian and Non-Semimartingale Noise Models

The standard linear quadratic Gaussian (LQG) framework assumes a Brownian noise process and relies on classical stochastic calculus tools, such as those based on It\^o calculus. In this paper, we solve a generalized linear quadratic optimal control problem where the process and measurement noises can be non-Markovian and non-semimartingale stochastic processes with sample paths that have low H\"older regularity. Since these noise models do not, in general, permit the use of the standard It\^o calculus, we employ rough path theory to formulate and solve the problem. By leveraging signature representations and controlled rough paths, we derive the optimal state estimation and control strategies.

eess.SY↗

arXiv2025

Control-Oriented System Identification: Classical, Learning, and Physics-Informed Approaches

We survey classical, machine learning, and data-driven system identification approaches to learn control-relevant and physics-informed models of dynamical systems. Recently, machine learning approaches have enabled system identification from noisy, high-dimensional, and complex data. However, their utility is limited by their ability to provide provable guarantees on control-relevant properties. Meanwhile, control theory has identified several properties that are useful in analysis and control synthesis, such as dissipativity, monotonicity, energy conservation, and symmetry-preserving structures. We posit that merging system identification with such control-relevant or physics-informed properties can provide useful inductive bias, enhance explainability, enable control synthesis with provable guarantees, and improve sample complexity. We formulate system identification as an optimization problem where control-relevant properties can be enforced through direct parameterization (constraining the model structure to satisfy a desired property by construction), soft constraints (encouraging control-relevant properties through regularization or penalty terms), and hard constraints (imposing control-relevant properties as constraints in the optimization problem). Through this lens, we survey methods to learn physics-informed and control-relevant models spanning classical linear and nonlinear system identification, machine learning approaches, and direct identification through data-driven and behavioral representations. We also provide several expository examples that are accompanied by code and brief tutorials on a public Github repository. We also describe challenging directions for future research, including identification in networked, switched, and time-varying systems, experiment design, and bridging the gaps between data-driven, learning-based, and control-oriented approaches.

eess.SY↗

arXiv2025

Harnessing Data from Clustered LQR Systems: Personalized and Collaborative Policy Optimization

It is known that reinforcement learning (RL) is data-hungry. To improve sample-efficiency of RL, it has been proposed that the learning algorithm utilize data from 'approximately similar' processes. However, since the process models are unknown, identifying which other processes are similar poses a challenge. In this work, we study this problem in the context of the benchmark Linear Quadratic Regulator (LQR) setting. Specifically, we consider a setting with multiple agents, each corresponding to a copy of a linear process to be controlled. The agents' local processes can be partitioned into clusters based on similarities in dynamics and tasks. Combining ideas from sequential elimination and zeroth-order policy optimization, we propose a new algorithm that performs simultaneous clustering and learning to output a personalized policy (controller) for each cluster. Under a suitable notion of cluster separation that captures differences in closed-loop performance across systems, we prove that our approach guarantees correct clustering with high probability. Furthermore, we show that the sub-optimality gap of the policy learned for each cluster scales inversely with the size of the cluster, with no additional bias, unlike in prior works on collaborative learning-based control. Our work is the first to reveal how clustering can be used in data-driven control to learn personalized policies that enjoy statistical gains from collaboration but do not suffer sub-optimality due to inclusion of data from dissimilar processes. From a distributed implementation perspective, our method is attractive as it incurs only a mild logarithmic communication overhead.

cs.LG↗

arXiv2025

Heterogeneous Multi-Agent Task-Assignment with Uncertain Execution Times and Preferences

While sequential task assignment for a single agent has been widely studied, such problems in a multi-agent setting, where the agents have heterogeneous task preferences or capabilities, remain less well-characterized. We study a multi-agent task assignment problem where a central planner assigns recurring tasks to multiple members of a team over a finite time horizon. For any given task, the members have heterogeneous capabilities in terms of task completion times, task resource consumption (which can model variables such as energy or attention), and preferences in terms of the rewards they collect upon task completion. We assume that the reward, execution time, and resource consumption for each member to complete any task are stochastic with unknown distributions. The goal of the planner is to maximize the total expected reward that the team receives over the problem horizon while ensuring that the resource consumption required for any assigned task is within the capability of the agent. We propose and analyze a bandit algorithm for this problem. Since the bandit algorithm relies on solving an optimal task assignment problem repeatedly, we analyze the achievable regret in two cases: when we can solve the optimal task assignment exactly and when we can solve it only approximately.

cs.MA↗

arXiv2025

Parameter-Free Federated TD Learning with Markov Noise in Heterogeneous Environments

Federated learning (FL) can dramatically speed up reinforcement learning by distributing exploration and training across multiple agents. It can guarantee an optimal convergence rate that scales linearly in the number of agents, i.e., a rate of $\tilde{O}(1/(NT)),$ where $T$ is the iteration index and $N$ is the number of agents. However, when the training samples arise from a Markov chain, existing results on TD learning achieving this rate require the algorithm to depend on unknown problem parameters. We close this gap by proposing a two-timescale Federated Temporal Difference (FTD) learning with Polyak-Ruppert averaging. Our method provably attains the optimal $\tilde{O}(1/NT)$ rate in both average-reward and discounted settings--offering a parameter-free FTD approach for Markovian data. Although our results are novel even in the single-agent setting, they apply to the more realistic and challenging scenario of FL with heterogeneous environments.

cs.LG↗

arXiv2025

Communication-aware Wide-Area Damping Control using Risk-Constrained Reinforcement Learning

Non-ideal communication links, especially delays, critically affect fast networked controls in power systems, such as the wide-area damping control (WADC). Traditionally, a delay estimation and compensation approach is adopted to address this cyber-physical coupling, but it demands very high accuracy for the fast WADC and cannot handle other cyber concerns like link failures or {cyber perturbations}. Hence, we propose a new risk-constrained framework that can target the communication delays, yet amenable to general uncertainty under the cyber-physical couplings. Our WADC model includes the synchronous generators (SGs), and also voltage source converters (VSCs) for additional damping capabilities. To mitigate uncertainty, a mean-variance risk constraint is introduced to the classical optimal control cost of the linear quadratic regulator (LQR). Unlike estimating delays, our approach can effectively mitigate large communication delays by improving the worst-case performance. A reinforcement learning (RL)-based algorithm, namely, stochastic gradient-descent with max-oracle (SGDmax), is developed to solve the risk-constrained problem. We further show its guaranteed convergence to stationarity at a high probability, even using the simple zero-order policy gradient (ZOPG). Numerical tests on the IEEE 68-bus system not only verify SGDmax's convergence and VSCs' damping capabilities, but also demonstrate that our approach outperforms conventional delay compensator-based methods under estimation error. While focusing on performance improvement under large delays, our proposed risk-constrained design can effectively mitigate the worst-case oscillations, making it equally effective for addressing other communication issues and cyber perturbations.

eess.SY↗

arXiv2025

The role of communication delays in the optimal control of spatially invariant systems

We study optimal proportional feedback controllers for spatially invariant systems when the controller has access to delayed state measurements received from different spatial locations. We analyze how delays affect the spatial locality of the optimal feedback gain leveraging the problem decoupling in the spatial frequency domain. For the cases of expensive control and small delay, we provide exact expressions of the optimal controllers in the limit for infinite control weight and vanishing delay, respectively. In the expensive control regime, the optimal feedback control law decomposes into a delay-aware filtering of the delayed state and the optimal controller in the delay-free setting. Under small delays, the optimal controller is a perturbation of the delay-free one which depends linearly on the delay. We illustrate our analytical findings with a reaction-diffusion process over the real line and a multi-agent system coupled through circulant matrices, showing that delays reduce the effectiveness of optimal feedback control and may require each subsystem within a distributed implementation to communicate with farther-away locations.

math.OC↗