Searcharxiv⌕ Search

arXiv subjects

Pavlo O. Kasyanov

Publications and source records attributed to Pavlo O. Kasyanov.

At least 19 recordsLinked to original sources
arXiv2025

Drone Swarm Energy Management

This note presents an analytical framework for decision-making in drone swarm systems operating under uncertainty, based on the integration of Partially Observable Markov Decision Processes (POMDP) with Deep Deterministic Policy Gradient (DDPG) reinforcement learning. The proposed approach enables adaptive control and cooperative behavior of unmanned aerial vehicles (UAVs) within a cognitive AI platform, where each agent learns optimal energy management and navigation policies from dynamic environmental states. We extend the standard DDPG architecture with a belief-state representation derived from Bayesian filtering, allowing for robust decision-making in partially observable environments. In this paper, for the Gaussian case, we numerically compare the performance of policies derived from DDPG to optimal policies for discretized versions of the original continuous problem. Simulation results demonstrate that the POMDP-DDPG-based swarm control model significantly improves mission success rates and energy efficiency compared to baseline methods. The developed framework supports distributed learning and decision coordination across multiple agents, providing a foundation for scalable cognitive swarm autonomy. The outcomes of this research contribute to the advancement of energy-aware control algorithms for intelligent multi-agent systems and can be applied in security, environmental monitoring, and infrastructure inspection scenarios.

math.OC↗
arXiv2025

Continuity of Filters for Discrete-Time Control Problems Defined by Explicit Equations

Discrete time control systems whose dynamics and observations are described by stochastic equations are common in engineering, operations research, health care, and economics. For example, stochastic filtering problems are usually defined via stochastic equations. These problems can be reduced to Markov decision processes (MDPs) whose states are posterior state distributions, and transition probabilities for such MDPs are sometimes called filters. This paper investigates sufficient conditions on transition and observation functions for the original problems to guarantee weak continuity of the filter. Under mild conditions on cost functions, weak continuity implies the existence of optimal policies minimizing the expected total costs, the validity of optimality equations, and convergence of value iterations to optimal values. This paper uses recent results on weak continuity of filters for partially observable MDPs defined by transition and observation probabilities. It develops a criterion of weak continuity of transition probabilities and a sufficient condition for continuity in total variation of transition probabilities. The results are illustrated with applications to filtering problems.

math.OC↗
arXiv2025

Average-Cost MDPs with Infinite State and Action Sets: New Sufficient Conditions for Optimality Inequalities and Equations

This paper studies discrete-time average-cost infinite-horizon Markov decision processes (MDPs) with Borel state and action sets. It introduces new sufficient conditions for { the} validity of optimality inequalities and optimality equations for MDPs with weakly and setwise continuous transition probabilities. These inequalities and equations imply the existence of deterministic optimal policies.

math.OC↗
arXiv2025

Chain recurrence and structure of omega-limit sets of multivalued semiflows

We study properties of !-limit sets of multivalued semiflows like chain recurrence or the existence of cyclic chains. First, we prove that under certain conditions the omega-limit set of a trajectory is chain recurrent, applying this result to an evolution differential inclusion with upper semicontinous right-hand side. Second, we give conditions ensuring that the omega-limit set of a trajectory contains a cyclic chain. Using this result we are able to check that the omega-limit set of every trajectory of a reaction-diffusion equation without uniqueness of solutions is an equilibrium.

math.DS↗
arXiv2024

On Right Continuity in $(L^2)^3$ at All the Points of Energy-regularized Solutions for the 3D Navier-Stokes Equations

In this note I provide the notion of energy-regularized solutions (ER-solutions) of the 3D Navier-Stokes equations. These solutions can be obtained via the standard Galerkin arguments. I prove that each ER-solution for the 3D Navier-Stokes system satisfies Leray-Hopf property. Moreover, each ER-solution is rightly continuous in the standard phase space $H$ endowed with the strong convergence topology.

math.AP↗
arXiv2023

Equivalent Conditions for Weak Continuity of Nonlinear Filters

This paper studies weak continuity of nonlinear filters. It is well-known that Borel measurability of transition probabilities for problems with incomplete state observations is preserved when the original discrete-time process is replaced with the process whose states are belief probabilities. It is also known that the similar preservation may not hold for weak continuity of transition probabilities. In this paper we show that the sufficient condition for weak continuity of transition probabilities for beliefs introduced by Kara, Saldi, and Yuksel (2019) is a necessary and sufficient condition for semi-uniform Feller continuity of transition probabilities. The property of semi-uniform Feller continuity was introduced in Feinberg, Kasyanov, and Zgurovsky (2021), and, if the original transition probability has this property, then the transition probability of the process, whose state is a pair consisting of the belief probability and observation, also has this property. Thus, this property implies weak continuity of nonlinear filters. This paper also reviews several necessary and sufficient conditions for semi-uniform Feller continuity.

math.OC↗
arXiv2023

Semi-Uniform Feller Stochastic Kernels

This paper studies transition probabilities from a Borel subset of a Polish space to a product of two Borel subsets of Polish spaces. For such transition probabilities it introduces and studies the property of semi-uniform Feller continuity. This paper provides several equivalent definitions of semi-uniform Feller continuity and establishes its preservation under integration. The motivation for this study came from the theory of Markov decision processes with incomplete information, and this paper provides fundamental results useful for this theory.

math.PR↗
arXiv2022

Markov Decision Processes with Incomplete Information and Semi-Uniform Feller Transition Probabilities

This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important feature of these models is that their classic reduction to Completely Observable Markov Decision Processes with belief states preserves semi-uniform Feller continuity of transition probabilities. Under mild assumptions on cost functions, optimal policies exist, optimality equations hold, and value iterations converge to optimal values for these models. In particular, for Partially Observable Markov Decision Processes the results of this paper imply new and generalize several known sufficient conditions on transition and observation probabilities for weak continuity of transition probabilities for Markov Decision Processes with belief states, the existence of optimal policies, validity of optimality equations defining optimal policies, and convergence of value iterations to optimal values.

math.OC↗
arXiv2022

Epi-Convergence of Expectation Functions under Varying Measures and Integrands

For expectation functions on metric spaces, we provide sufficient conditions for epi-convergence under varying probability measures and integrands, and examine applications in the area of sieve estimators, mollifier smoothing, PDE-constrained optimization, and stochastic optimization with expectation constraints. As a stepping stone to epi-convergence of independent interest, we develop parametric Fatou's lemmas under mild integrability assumptions. In the setting of Suslin metric spaces, the assumptions are expressed in terms of Pasch-Hausdorff envelopes. For general metric spaces, the assumptions shift to semicontinuity of integrands also on the sample space, which then is assumed to be a metric space.

math.OC↗
arXiv2021

Solutions for Zero-Sum Two-Player Games with Noncompact Decision Sets and Unbounded Payoffs

This paper provides sufficient conditions for the existence of solutions for two-person zero-sum games with inf/sup-compact payoff functions and with possibly noncompact decision sets for both players. Payoff functions may be unbounded, and we do not assume any convexity/concavity-type conditions. For such games expected payoff may not exist for some pairs of strategies. The results of this paper imply several classic facts. The paper also provides sufficient conditions for the existence of a value and solutions for each player. The results of this paper are illustrated with the number guessing game.

math.OC↗
arXiv2021

A Class of Solvable Markov Decision Models with Incomplete Information

This paper investigates natural conditions for the existence of optimal policies for a Markov decision process with incomplete information (MDPII) and with expected total costs. The MDPII is the classic model of a controlled stochastic process with incomplete state observations which is more general than Partially Observable Markov Decision Processes (POMDPs). For MDPIIs we introduce the notion of a semi-uniform Feller transition probability, which is stronger than the notion of a weakly continuous transition probability. We show that an MDPII has a semi-uniform Feller transition probability if and only if the corresponding belief MDP also has a semi-uniform Feller transition probability. This fact has several corollaries. In particular, it provides new and implies all known sufficient conditions for the existence of optimal policies for POMDPs with expected total costs

math.OC↗
arXiv2021

Continuity of Parametric Optima for Possibly Discontinuous Functions and Noncompact Decision Sets

This paper investigates continuity properties of value functions and solutions for parametric optimization problems. These problems are important in operations research, control, and economics because optimality equations are their particular cases. The classic fact, Berge's maximum theorem, gives sufficient conditions for continuity of value functions and upper semicontinuity of solution multifunctions. Berge's maximum theorem assumes that the objective function is continuous and the multifunction of feasible sets is compact-valued. These assumptions are not satisfied in many applied problems, which historically has limited the relevance of the theorem. This paper generalizes Berge's maximum theorem in three directions: (i) the objective function may not be continuous, (ii) the multifunction of feasible sets may not be compact-valued, and (iii) necessary and sufficient conditions are provided. To illustrate the main theorem, this paper provides applications to inventory control and to the analysis of robust optimization over possibly noncompact action sets and discontinuous objective functions.

math.OC↗
arXiv2021

Average Cost Markov Decision Processes with Semi-Uniform Feller Transition Probabilities

This paper studies average-cost Markov decision processes with semi-uniform Feller transition probabilities. This class of MDPs was recently introduced by the authors to study MDPs with incomplete information. This paper studies the validity of optimality inequalities, the existence of optimal policies, and the approximations of optimal policies by policies optimizing total discounted costs.

math.OC↗
arXiv2021

MDPs with Setwise Continuous Transition Probabilities

This paper describes the structure of optimal policies for infinite-state Markov Decision Processes with setwise continuous transition probabilities. The action sets may be noncompact. The objective criteria are either the expected total discounted and undiscounted costs or average costs per unit time. The analysis of optimality equations and inequalities is based on the optimal selection theorem for inf-compact functions introduced in this paper.

math.OC↗
arXiv2019

Fatou's Lemma in Its Classic Form and Lebesgue's Convergence Theorems for Varying Measures with Applications to MDPs

The classic Fatou lemma states that the lower limit of a sequence of integrals of functions is greater or equal than the integral of the lower limit. It is known that Fatou's lemma for a sequence of weakly converging measures states a weaker inequality because the integral of the lower limit is replaced with the integral of the lower limit in two parameters, where the second parameter is the argument of the functions. This paper provides sufficient conditions when Fatou's lemma holds in its classic form for a sequence of weakly converging measures. The functions can take both positive and negative values. The paper also provides similar results for sequences of setwise converging measures. It also provides Lebesgue's and monotone convergence theorems for sequences of weakly and setwise converging measures. The obtained results are used to prove broad sufficient conditions for the validity of optimality equations for average-cost Markov decision processes.

math.PR↗
arXiv2019

Fatou's Lemma for Weakly Converging Measures under the Uniform Integrability Condition

This note describes Fatou's lemma and Lebesgue's dominated convergence theorem for a sequence of measures converging weakly to a finite measure and for a sequence of functions whose negative parts are uniformly integrable with respect to these measures. The note also provides new formulations of uniform Fatou's lemma, uniform Lebesgue convergence theorem, the Dunford-Pettis theorem, and the fundamental theorem for Young measures based on the equivalence of uniform integrability and the apparently weaker property of asymptotic uniform integrability for sequences of functions and finite measures.

math.CA↗
arXiv2018

An example showing that A-lower semi-continuity is essential for minimax continuity theorems

Recently Feinberg et al. [arXiv:1609.03990] established results on continuity properties of minimax values and solution sets for a function of two variables depending on a parameter. Such minimax problems appear in games with perfect information, when the second player knows the move of the first one, in turn-based games, and in robust optimization. Some of the results in [arXiv:1609.03990] are proved under the assumption that the multifunction, defining the domains of the second variable, is $A$-lower semi-continuous. The $A$-lower semi-continuity property is stronger than lower semi-continuity, but in several important cases these properties coincide. This note provides an example demonstrating that in general the $A$-lower semi-continuity assumption cannot be relaxed to lower semi-continuity.

math.OC↗