arXiv subjects

Melkior Ornik

Publications and source records attributed to Melkior Ornik.

At least 91 records · Page 5Linked to original sources

arXiv2023

Deception in Supervisory Control

The use of deceptive strategies is important for an agent that attempts not to reveal his intentions in an adversarial environment. We consider a setting in which a supervisor provides a reference policy and expects an agent to follow the reference policy and perform a task. The agent may instead follow a different, deceptive policy to achieve a different task. We model the environment and the behavior of the agent with a Markov decision process, represent the tasks of the agent and the supervisor with reachability specifications, and study the synthesis of optimal deceptive policies for such agents. We also study the synthesis of optimal reference policies that prevent deceptive strategies of the agent and achieve the supervisor's task with high probability. We show that the synthesis of optimal deceptive policies has a convex optimization problem formulation, while the synthesis of optimal reference policies requires solving a nonconvex optimization problem. We also show that the synthesis of optimal reference policies is NP-hard.

math.OC↗

arXiv2022

Post-Disaster Repair Crew Assignment Optimization Using Minimum Latency

Across infrastructure domains, physical damage caused by storms and other weather events often requires costly and time-sensitive repairs to restore services as quickly as possible. While recent studies have used agent-based models to estimate the cost of repairs, the implemented strategies for assignment of repair crews to different locations are generally human-driven or based on simple rules. In order to find performant strategies, we continue with an agent-based model, but approach this problem as a combinational optimization problem known as the Minimum Weighted Latency Problem for multiple repair crews. We apply a partitioning algorithm that balances the assignment of targets amongst all the crews using two different heuristics that optimize either the importance of repair locations or the travel time between them. We benchmark our algorithm on both randomly generated graphs as well as data derived from a real-world urban environment, and show that our algorithm delivers significantly better assignments than existing methods.

cs.MA↗

arXiv2022

Quantitative Resilience of Linear Systems

Actuator malfunctions may have disastrous consequences for systems not designed to mitigate them. We focus on the loss of control authority over actuators, where some actuators are uncontrolled but remain fully capable. To counteract the undesirable outputs of these malfunctioning actuators, we use real-time measurements and redundant actuators. In this setting, a system that can still reach its target is deemed resilient. To quantify the resilience of a system, we compare the shortest time for the undamaged system to reach the target with the worst-case shortest time for the malfunctioning system to reach the same target, i.e., when the malfunction makes that time the longest. Contrary to prior work on driftless linear systems, the absence of analytical expression for time-optimal controls of general linear systems prevents an exact calculation of quantitative resilience. Instead, relying on Lyapunov theory we derive analytical bounds on the nominal and malfunctioning reach times in order to bound quantitative resilience. We illustrate our work on a temperature control system.

eess.SY↗

arXiv2022

Designing Resilient Linear Driftless Systems

Critical systems must be designed resilient to all kinds of malfunctions. We are especially interested by the loss of control authority over actuators. This malfunction considers actuators producing uncontrolled and possibly undesirable inputs. We investigate the design of resilient linear systems capable of reaching their target even after such a malfunction. In contrast with the settings of robust control and fault-tolerant control, we consider undesirable but observable inputs of the same magnitude as controls since they are produced by a faulty actuator of the system. The control inputs can then depend on these undesirable inputs. Building on our previous work, we focus on designing resilient systems able to withstand the loss of one or multiple actuators. Since resilience refers to the existence of a control law driving the state to the target, we naturally continue with the synthesis of such a control law. We conclude with a numerical application of our theory on the ADMIRE fighter jet model.

eess.SY↗

arXiv2022

Online Guaranteed Reachable Set Approximation for Systems with Changed Dynamics and Control Authority

This work presents a method of efficiently computing inner and outer approximations of forward reachable sets for nonlinear control systems with changed dynamics and diminished control authority, given an a priori computed reachable set for the nominal system. The method functions by shrinking or inflating a precomputed reachable set based on prior knowledge of the system's trajectory deviation growth dynamics, depending on whether an inner approximation or outer approximation is desired. These dynamics determine an upper bound on the minimal deviation between two trajectories emanating from the same point that are generated on the nominal system using nominal control inputs, and by the impaired system based on the diminished set of control inputs, respectively. The dynamics depend on the given Hausdorff distance bound between the nominal set of admissible controls and the possibly unknown impaired space of admissible controls, as well as a bound on the rate change between the nominal and off-nominal dynamics. Because of its computational efficiency compared to direct computation of the off-nominal reachable set, this procedure can be applied to on-board fault-tolerant path planning and failure recovery. In addition, the proposed algorithm does not require convexity of the reachable sets unlike our previous work, thereby making it suitable for general use. We raise a number of implementational considerations for our algorithm, and we present three illustrative examples, namely an application to the heading dynamics of a ship, a lower triangular dynamical system, and a system of coupled linear subsystems.

math.OC↗

arXiv2022

Lodestar: An Integrated Embedded Real-Time Control Engine

In this work we present Lodestar, an integrated engine for rapid real-time control system development. Using a functional block diagram paradigm, Lodestar allows for complex multi-disciplinary control software design, while automatically resolving execution order, circular data-dependencies, and networking. In particular, Lodestar presents a unified set of control, signal processing, and computer vision routines to users, which may be interfaced with external hardware and software packages using interoperable user-defined wrappers. Lodestar allows for user-defined block diagrams to be directly executed, or for them to be translated to overhead-free source code for integration in other programs. We demonstrate how our framework departs from approaches used in state-of-the-art simulation frameworks to enable real-time performance, and compare its capabilities to existing solutions in the realm of control software. To demonstrate the utility of Lodestar in real-time control systems design, we have applied Lodestar to implement two real-time torque-based controller for a robotic arm. In addition, we have developed a novel autofocus algorithm for use in thermography-based localization and parameter estimation in electrosurgery and other areas of robot-assisted surgery. We compare our algorithm design approach in Lodestar to a classical ground-up approach, showing that Lodestar considerably eases the design process. We also show how Lodestar can seamlessly interface with existing simulation and networking framework in a number of simulation examples.

cs.RO↗

arXiv2022

Distributed Transient Safety Verification via Robust Control Invariant Sets: A Microgrid Application

Modern safety-critical energy infrastructures are increasingly operated in a hierarchical and modular control framework which allows for limited data exchange between the modules. In this context, it is important for each module to synthesize and communicate constraints on the values of exchanged information in order to assure system-wide safety. To ensure transient safety in inverter-based microgrids, we develop a set invariance-based distributed safety verification algorithm for each inverter module. Applying Nagumo's invariance condition, we construct a robust polynomial optimization problem to jointly search for safety-admissible set of control set-points and design parameters, under allowable disturbances from neighbors. We use sum-of-squares (SOS) programming to solve the verification problem and we perform numerical simulations using grid-forming inverters to illustrate the algorithm.

eess.SY↗

arXiv2022

Space Exploration Architecture and Design Framework for Commercialization

The trend of space commercialization is changing the decision-making process for future space exploration architectures, and there is a growing need for a new decision-making framework that explicitly considers the interactions between the mission coordinator (i.e., government) and the commercial players. In response to this challenge, this paper develops a framework for space exploration and logistics decision making that considers the incentive mechanism to stimulate commercial participation in future space infrastructure development and deployment. By extending the state-of-the-art space logistics design formulations from the game-theoretic perspective, the relationship between the mission coordinator and commercial players is first analyzed, and then the formulation for the optimal architecture design and incentive mechanism in three different scenarios is derived. To demonstrate and evaluate the effectiveness of the proposed framework, a case study on lunar habitat infrastructure design and deployment is conducted. Results show how total mission demands and in-situ resource utilization system performances after deployment may impact the cooperation among stakeholders. As an outcome of this study, an incentive-based decision-making framework that can benefit both the mission coordinator and the commercial players from commercialization is derived, leading to a mutually beneficial space exploration between the government and the industry.

math.OC↗

arXiv2021

The Maximax Minimax Quotient Theorem

We present an optimization problem emerging from optimal control theory and situated at the intersection of fractional programming and linear max-min programming on polytopes. A naïve solution would require solving four nested, possibly nonlinear, optimization problems. Instead, relying on numerous geometric arguments we determine an analytical solution to this problem. In the course of proving our main theorem we also establish another optimization result stating that the minimum of a specific minimax optimization is located at a vertex of the constraint set.

math.OC↗

arXiv2021

Reachability of Nonlinear Systems with Unknown Dynamics

Determining the reachable set for a given nonlinear control system is crucial for system control and planning. However, computing such a set is impossible if the system's dynamics are not fully known. This paper is motivated by a scenario where a system suffers an adverse event mid-operation, resulting in a substantial change to the system's dynamics, rendering them largely unknown. Our objective is to conservatively approximate the system's reachable set solely from its local dynamics at a single point and the bounds on the rate of change of its dynamics. We translate this knowledge about the system dynamics into an ordinary differential inclusion. We then derive a conservative approximation of the velocities available to the system at every system state. An inclusion using this approximation can be interpreted as a control system; the trajectories of the derived control system are guaranteed to be trajectories of the unknown system. To illustrate the practical implementation and consequences of our work, we apply our algorithm to a simplified model of an unmanned aerial vehicle.

math.OC↗

arXiv2021

Inapplicability of the TVOR Method to USHMM Data Outlier Identification

Recent paper "TVOR: Finding Discrete Total Variation Outliers Among Histograms" [arXiv:2012.11574] introduces the Total Variation Outlier Recognizer (TVOR) method for identification of outliers among a given set of histograms. After providing a theoretical discussion of the method and verifying its success on synthetic and population census data, it applies the TVOR model to histograms of ages of Holocaust victims produced using United States Holocaust Memorial Museum data. It purports to identify the list of victims of the Jasenovac concentration camp as potentially suspicious. In this comment paper, we show that the TVOR model and its assumptions are grossly inapplicable to the considered dataset. When applied to the considered data, the model is biased in assigning a higher outlier score to histograms of larger sizes, the set of data points is extremely sparse around the point of interest, the dataset has not been reviewed to remove obvious data processing errors, and, contrary to the model requirements, the distributions of the victims' ages naturally vary significantly across victim lists.

stat.ME↗

arXiv2021

Efficient Strategy Synthesis for MDPs with Resource Constraints

We consider qualitative strategy synthesis for the formalism called consumption Markov decision processes. This formalism can model dynamics of an agents that operates under resource constraints in a stochastic environment. The presented algorithms work in time polynomial with respect to the representation of the model and they synthesize strategies ensuring that a given set of goal states will be reached (once or infinitely many times) with probability 1 without resource exhaustion. In particular, when the amount of resource becomes too low to safely continue in the mission, the strategy changes course of the agent towards one of a designated set of reload states where the agent replenishes the resource to full capacity; with sufficient amount of resource, the agent attempts to fulfill the mission again. We also present two heuristics that attempt to reduce expected time that the agent needs to fulfill the given mission, a parameter important in practical planning. The presented algorithms were implemented and numerical examples demonstrate (i) the effectiveness (in terms of computation time) of the planning approach based on consumption Markov decision processes and (ii) the positive impact of the two heuristics on planning in a realistic example.

cs.AI↗

arXiv2021

Quantitative Resilience of Linear Driftless Systems

This paper introduces the notion of quantitative resilience of a control system. Following prior work, we study systems enduring a loss of control authority over some of their actuators. Such a malfunction results in actuators producing possibly undesirable inputs over which the controller has real-time readings but no control. By definition, a system is resilient if it can still reach a target after a loss of control authority. However, after a malfunction a resilient system might be significantly slower to reach a target compared to its initial capabilities. We quantify this loss of performance through the new concept of quantitative resilience. We define this metric as the maximal ratio of the minimal times required to reach any target for the initial and malfunctioning systems. Naïve computation of quantitative resilience directly from the definition is a time-consuming task as it requires solving four nested, possibly nonlinear, optimization problems. The main technical contribution of this work is to provide an efficient method to compute quantitative resilience. Relying on control theory and on three novel geometric results we reduce the computation of quantitative resilience to a single linear optimization problem. We illustrate our method on two numerical examples: an opinion dynamics scenario and a trajectory controller for low-thrust spacecrafts.

eess.SY↗

arXiv2021

Learning and Planning for Time-Varying MDPs Using Maximum Likelihood Estimation

This paper proposes a formal approach to online learning and planning for agents operating in a priori unknown, time-varying environments. The proposed method computes the maximally likely model of the environment, given the observations about the environment made by an agent earlier in the system run and assuming knowledge of a bound on the maximal rate of change of system dynamics. Such an approach generalizes the estimation method commonly used in learning algorithms for unknown Markov decision processes with time-invariant transition probabilities, but is also able to quickly and correctly identify the system dynamics following a change. Based on the proposed method, we generalize the exploration bonuses used in learning for time-invariant Markov decision processes by introducing a notion of uncertainty in a learned time-varying model, and develop a control policy for time-varying Markov decision processes based on the exploitation and exploration trade-off. We demonstrate the proposed methods on four numerical examples: a patrolling task with a change in system dynamics, a two-state MDP with periodically changing outcomes of actions, a wind flow estimation task, and a multi-armed bandit problem with periodically changing probabilities of different rewards.

cs.LG↗

arXiv2020

Guaranteed Reachability for Systems with Unknown Dynamics

The problem of computing the reachable set for a given system is a quintessential question in nonlinear control theory. While previous work has yielded a plethora of approximate and analytical methods for determining such a set, these methods naturally require the knowledge of the controlled system dynamics throughout the state space. In contrast to such classical methods, this paper considers the question of estimating the reachable set of a control system using only the knowledge of local system dynamics at a single point and a bound on the rate of change of dynamics. Namely, motivated by the need for safety-critical planning for systems with unknown dynamics, we consider the problem of describing the guaranteed reachability set: the set of all states that are guaranteed to be reachable regardless of the true system dynamics, given the current knowledge about the system. We show that such a set can be underapproximated by a reachable set of a related known system whose dynamics at every state depend on the velocity vectors that are guaranteed to all control systems consistent with the assumed knowledge. Complementing the theory, numerical examples of a single-dimensional control system and a simple model of an aircraft in distress verify that such an underapproximation is meaningful in practice, and may indeed equal the desired guaranteed reachability set.

math.OC↗

arXiv2020

Qualitative Controller Synthesis for Consumption Markov Decision Processes

Consumption Markov Decision Processes (CMDPs) are probabilistic decision-making models of resource-constrained systems. In a CMDP, the controller possesses a certain amount of a critical resource, such as electric power. Each action of the controller can consume some amount of the resource. Resource replenishment is only possible in special reload states, in which the resource level can be reloaded up to the full capacity of the system. The task of the controller is to prevent resource exhaustion, i.e. ensure that the available amount of the resource stays non-negative, while ensuring an additional linear-time property. We study the complexity of strategy synthesis in consumption MDPs with almost-sure Büchi objectives. We show that the problem can be solved in polynomial time. We implement our algorithm and show that it can efficiently solve CMDPs modelling real-world scenarios.

cs.FL↗

arXiv2019

Entropy Maximization for Markov Decision Processes Under Temporal Logic Constraints

We study the problem of synthesizing a policy that maximizes the entropy of a Markov decision process (MDP) subject to a temporal logic constraint. Such a policy minimizes the predictability of the paths it generates, or dually, maximizes the exploration of different paths in an MDP while ensuring the satisfaction of a temporal logic specification. We first show that the maximum entropy of an MDP can be finite, infinite or unbounded. We provide necessary and sufficient conditions under which the maximum entropy of an MDP is finite, infinite or unbounded. We then present an algorithm which is based on a convex optimization problem to synthesize a policy that maximizes the entropy of an MDP. We also show that maximizing the entropy of an MDP is equivalent to maximizing the entropy of the paths that reach a certain set of states in the MDP. Finally, we extend the algorithm to an MDP subject to a temporal logic specification. In numerical examples, we demonstrate the proposed method on different motion planning scenarios and illustrate the relation between the restrictions imposed on the paths by a specification, the maximum entropy, and the predictability of paths.

math.OC↗

arXiv2019

Incentive Design for Temporal Logic Objectives

We study the problem of designing an optimal sequence of incentives that a principal should offer to an agent so that the agent's optimal behavior under the incentives realizes the principal's objective expressed as a temporal logic formula. We consider an agent with a finite decision horizon and model its decision-making process as a Markov decision process (MDP). Under certain assumptions, we present a polynomial-time algorithm to synthesize an incentive sequence that minimizes the cost to the principal. We show that if the underlying MDP has only deterministic transitions, the principal can hide its objective from the agent and still realize the desired behavior through incentives. On the other hand, an MDP with stochastic transitions may require the principal to share its objective with the agent. Finally, we demonstrate the proposed method in motion planning examples where a principal changes the optimal trajectory of an agent by providing incentives.

math.OC↗