arXiv subjects

Stefan Streif

Publications and source records attributed to Stefan Streif.

71 records · Page 4Linked to original sources

arXiv2020

Stacked adaptive dynamic programming with unknown system model

Adaptive dynamic programming is a collective term for a variety of approaches to infinite-horizon optimal control. Common to all approaches is approximation of the infinite-horizon cost function based on dynamic programming philosophy. Typically, they also require knowledge of a dynamical model of the system. In the current work, application of adaptive dynamic programming to a system whose dynamical model is unknown to the controller is addressed. In order to realize the control algorithm, a model of the system dynamics is estimated with a Kalman filter. A stacked control scheme to boost the controller performance is suggested. The functioning of the new approach was verified in simulation and compared to the baseline represented by gradient descent on the running cost.

math.OC↗

arXiv2020

Nonsmooth stabilization and its computational aspects

This work has the goal of briefly surveying some key stabilization techniques for general nonlinear systems, for which, as it is well known, a smooth control Lyapunov function may fail to exist. A general overview of the situation with smooth and nonsmooth stabilization is provided, followed by a concise summary of basic tools and techniques, including general stabilization, sliding-mode control and nonsmooth backstepping. Their presentation is accompanied with examples. The survey is concluded with some remarks on computational aspects related to determination of sampling times and control actions.

math.OC↗

arXiv2020

A reinforcement learning method with closed-loop stability guarantee

Reinforcement learning (RL) in the context of control systems offers wide possibilities of controller adaptation. Given an infinite-horizon cost function, the so-called critic of RL approximates it with a neural net and sends this information to the controller (called "actor"). However, the issue of closed-loop stability under an RL-method is still not fully addressed. Since the critic delivers merely an approximation to the value function of the corresponding infinite-horizon problem, no guarantee can be given in general as to whether the actor's actions stabilize the system. Different approaches to this issue exist. The current work offers a particular one, which, starting with a (not necessarily smooth) control Lyapunov function (CLF), derives an online RL-scheme in such a way that practical semi-global stability property of the closed-loop can be established. The approach logically continues the work of the authors on parameterized controllers and Lyapunov-like constraints for RL, whereas the CLF now appears merely in one of the constraints of the control scheme. The analysis of the closed-loop behavior is done in a sample-and-hold (SH) manner thus offering a certain insight into the digital realization. The case study with a non-holonomic integrator shows the capabilities of the derived method to optimize the given cost function compared to a nominal stabilizing controller.

math.OC↗

arXiv2020

A Hierarchical Architecture for the Coordination of an Ensemble of Steam Generators

This work presents a hierarchical architecture for the optimal management of an ensemble of steam generators, which needs to jointly sustain a common load. The coordination of independent subsystems is provided by a multi-layer control scheme. A high-level optimizer computes the optimal shares of production to be allocated to single generators. At medium level, a robust tube-based model predictive control (MPC) is proposed to track the time-varying demand of the ensemble using a centralized, but aggregated model, whose order does not scale with the number of subsystems. At low level, decentralized controllers are in place to stabilize the internal boiler pressure. The control architecture enables the dynamic modification of the ensemble configuration and plug and play operations. Simulation results are reported to demonstrate the potentialities of the proposed approach.

eess.SY↗

arXiv2020

Model Predictive Control of a Food Production Unit: A Case Study for Lettuce Production

Plant factories with artificial light are widely researched for food production in a controlled environment. For such control tasks, models of the energy and resource exchange in the production unit as well as those of the plant's growth process may be used. To achieve minimal operation cost, optimal control strategies can be applied to the system, taking into account the availability of resources by control reference specification. A particular advantage of model predictive control (MPC) is the incorporation of constraints that comply with actuator limitations and general plant growth conditions. In this work, a model of a production unit is derived including a description of the relation between the actuators' electrical signals and the input values to the model. Furthermore, a preliminary model based state tracking control is evaluated for production unit containing Lettuce. It could be observed that the controller is capable to track the reference while satisfying the constraint under changing weather conditions and resource availability.

eess.SY↗

arXiv2020

Model predictive control with stage cost shaping inspired by reinforcement learning

This work presents a suboptimality study of a particular model predictive control with a stage cost shaping based on the ideas of reinforcement learning. The focus of the suboptimality study is to derive quantities relating the infinite-horizon cost function under the said variant of model predictive control to the respective infinite-horizon value function. The basis control scheme involves usual stabilizing constraints comprising of a terminal set and a terminal cost in the form of a local Lyapunov function. The stage cost is adapted using the principles of Q-learning, a particular approach to reinforcement learning. The work is concluded by case studies with two systems for wide ranges of initial conditions.

math.OC↗

arXiv2020

Maintaining Hard Infection Caps in Epidemics via the Theory of Barriers

Research in epidemiology often focusses on designing interventions that result in the number of infected individuals asymptotically approaching zero, without considering that this number may peak at high values during transients. Recent research has shown that a set-based approach could be used to address the problem, and we build on this idea by applying the theory of barriers to construct admissible and invariant sets for an epidemic model. We describe how these sets may be used to choose intervention strategies that maintain infection caps during epidemics. We also derive algebraic conditions of the model parameters that classify a system as being either comfortable, comfortable-viable, viable, or desperate.

math.OC↗

arXiv2020

Sustainability Analysis of Interconnected Food Production Systems via Theory of Barriers

Controlled environment agriculture (CEA) is used for efficient food production. Efficiency can be increased further by interconnecting different CEA systems (e.g. plants and insect larvae or fish and larvae), using products and by-products of one system in the other. These interconnected systems define an overall system that can be described by models of interacting species. It is necessary to identify system parameters (e.g. initial species concentration, harvest rate, feed quality, etc.) such that the resources are not exhausted. For such systems with interacting species, modelled by the Lotka-Volterra equations, a set-based approach based on the recent results of the theory of barriers to exactly determine the so-called admissible set (also known as viability kernel) and the maximal robust positively invariant set is presented. Using an example of a larvae-fish based production system, steps to obtain special trajectories which are the boundaries of the admissible set are shown. This admissible set is used to prevent the under and over population of the species in the CEA. Furthermore, conditions of the system parameters are stated, such that the existence of these trajectories can be guaranteed.

math.OC↗

arXiv2019

Design and validation of a low cost programmable controlled environment for study and production of plants, mushroom, and insect larvae

Use of commercial growth chambers for study of biological processes involved in biomass growth and production pose certain limitations on the nature of studies that can be performed in them. Optimization of biomass rearing and production process requires quantitative study of environment influences on the organism and eventually the products and byproducts consumed and produced. This work presents a low cost modular system designed to facilitate quantitative study of growth processes and resource exchanges in organisms such as plants, fungi and insect larvae. The proposed system constitutes of modular units each performing a specific function. A novel compact thermoelectric cooler based unit is designed for conditioning the air. Sensor cluster for measuring gas concentrations, air properties (temperature, humidity, pressure), and growing medium properties is implemented and tested. An actuator cluster for resource exchange and a wiring and control scheme for light spectrum adjustment is proposed. A three tier hierarchical software framework consisting of an open-source cloud platform for data aggregation and user interaction, embedded firmware for microcontroller, and an application development framework for test automation and experiment regime design is developed and presented. A series of experiments and tests were performed using the designed hardware and software to evaluate its capabilities and limitations. This controlled environment was used to study the photosynthesis and its dependency on temperature and light intensity in Ocimum basilicum. In a second experiment, evolution of metabolic activity of Hermetia illucens larvae over its larval phase was studied and the metabolic products and byproducts were quantitatively measured.

q-bio.QM↗

arXiv2018

Analysis of extremum value theorems for function spaces in optimal control under numerical uncertainty

The extremum value theorem for function spaces plays the central role in optimal control. It is known that computation of optimal control actions and policies is often prone to numerical errors which may be related to computability issues. The current work addresses a version of the extremum value theorem for function spaces under explicit consideration of numerical uncertainties. It is shown that certain function spaces are bounded in a suitable sense i.e. they admit finite approximations up to an arbitrary precision. The proof of this fact is constructive in the sense that it explicitly builds the approximating functions. Consequently, existence of approximate extremal functions is shown. Applicability of the theorem is investigated for finite--horizon optimal control, dynamic programming and adaptive dynamic programming. Some possible computability issues of the extremum value theorem in optimal control are shown on counterexamples

math.OC↗

arXiv2018

Practical sample-and-hold stabilization of nonlinear systems under approximate optimizers

It is a known fact that not all controllable systems can be asymptotically stabilized by a continuous static feedback. Several approaches have been developed throughout the last decades, including time-varying, dynamical and even discontinuous feedbacks. In the latter case, the sample-and-hold framework is widely used, in which the control input is held constant during sampling periods. Consequently, only practical stability can be achieved at best. Existing approaches often require solving optimization problems for finding stabilizing control actions exactly. In practice, each optimization routine has a finite accuracy which might influence the state convergence. This work shows, what bounds on optimization accuracy are required to achieve prescribed stability margins. Simulation studies support the claim that optimization accuracy has high influence on the state convergence.

math.OC↗

arXiv2016

A note on constructive treatment of eigenvectors

The eigenvalue problem plays a central role in linear algebra and its applications in control and optimization methods. In particular, many matrix decompositions rely upon computation of eigenvalue-eigenvector pairs, such as diagonal or Jordan normal forms. Unfortunately, numerical algorithms computing eigenvectors are prone to errors. Due to uncomputability of eigenpairs, perturbation theory and various regularization techniques only help if the matrix at hand possesses certain properties such as the absence of non-zero singular values, or the presence of a distinguishable gap between the large and small singular values. Posing such a requirement might be restrictive in some practical applications. In this note, we propose an alternative treatment of eigenvectors which is approximate and constructive. In comparison to classical eigenvectors whose computation is often prone to numerical instability, a constructive treatment allows addressing the computational uncertainty in a controlled way.

math.OC↗

arXiv2015

Stability for Receding-horizon Stochastic Model Predictive Control

A stochastic model predictive control (SMPC) approach is presented for discrete-time linear systems with arbitrary time-invariant probabilistic uncertainties and additive Gaussian process noise. Closed-loop stability of the SMPC approach is established by appropriate selection of the cost function. Polynomial chaos is used for uncertainty propagation through system dynamics. The performance of the SMPC approach is demonstrated using the Van de Vusse reactions.

eess.SY↗

arXiv2014

A Probabilistic Approach to Robust Optimal Experiment Design with Chance Constraints

Accurate estimation of parameters is paramount in developing high-fidelity models for complex dynamical systems. Model-based optimal experiment design (OED) approaches enable systematic design of dynamic experiments to generate input-output data sets with high information content for parameter estimation. Standard OED approaches however face two challenges: (i) experiment design under incomplete system information due to unknown true parameters, which usually requires many iterations of OED; (ii) incapability of systematically accounting for the inherent uncertainties of complex systems, which can lead to diminished effectiveness of the designed optimal excitation signal as well as violation of system constraints. This paper presents a robust OED approach for nonlinear systems with arbitrarily-shaped time-invariant probabilistic uncertainties. Polynomial chaos is used for efficient uncertainty propagation. The distinct feature of the robust OED approach is the inclusion of chance constraints to ensure constraint satisfaction in a stochastic setting. The presented approach is demonstrated by optimal experimental design for the JAK-STAT5 signaling pathway that regulates various cellular processes in a biological cell.

eess.SY↗

arXiv2014

Stochastic Nonlinear Model Predictive Control with Efficient Sample Approximation of Chance Constraints

This paper presents a stochastic model predictive control approach for nonlinear systems subject to time-invariant probabilistic uncertainties in model parameters and initial conditions. The stochastic optimal control problem entails a cost function in terms of expected values and higher moments of the states, and chance constraints that ensure probabilistic constraint satisfaction. The generalized polynomial chaos framework is used to propagate the time-invariant stochastic uncertainties through the nonlinear system dynamics, and to efficiently sample from the probability densities of the states to approximate the satisfaction probability of the chance constraints. To increase computational efficiency by avoiding excessive sampling, a statistical analysis is proposed to systematically determine a-priori the least conservative constraint tightening required at a given sample size to guarantee a desired feasibility probability of the sample-approximated chance constraint optimization problem. In addition, a method is presented for sample-based approximation of the analytic gradients of the chance constraints, which increases the optimization efficiency significantly. The proposed stochastic nonlinear model predictive control approach is applicable to a broad class of nonlinear systems with the sufficient condition that each term is analytic with respect to the states, and separable with respect to the inputs, states and parameters. The closed-loop performance of the proposed approach is evaluated using the Williams-Otto reactor with seven states, and ten uncertain parameters and initial conditions. The results demonstrate the efficiency of the approach for real-time stochastic model predictive control and its capability to systematically account for probabilistic uncertainties in contrast to a nonlinear model predictive control approaches.

math.OC↗

arXiv2013

Probabilistic and Set-based Model Invalidation and Estimation using LMIs

Probabilistic and set-based methods are two approaches for model invalidation, parameter and state estimation. Both classes of methods use different types of data, i.e. deterministic or probabilistic data, which allow different statements and applications. Ideally, however, all available data should be used in estimation and model invalidation methods. This paper presents an estimation and model invalidation framework combining set-based and probabilistically uncertain data for polynomial continuous-time systems. In particular, uncertain data on the moments and the support is used without the need to make explicit assumptions on the type of probability densities. The paper derives pointwise-in-time outer approximations of the moments of the probability densities associated with the states and parameters of the system. These approximations can be interpreted as guaranteed confidence intervals for the moment estimates. Furthermore, guaranteed bounds on the probability masses on subsets are derived and allow an estimation of the unknown probability densities. To calculate the estimates, the dynamics of the probability densities of the state trajectories are found by occupation measures of the nonlinear dynamics. This allows the construction of an infinite-dimensional linear program which incorporates the set- and moment-based data. This linear program is relaxed by a hierarchy of LMI problems providing, as shown elsewhere, an almost uniformly convergent sequence of outer approximations of the estimated sets. The approach is demonstrated with numerical examples.

math.OC↗

arXiv2013

Estimation of consistent parameter sets for continuous-time nonlinear systems using occupation measures and LMI relaxations

Obtaining initial conditions and parameterizations leading to a model consistent with available measurements or safety specifications is important for many applications. Examples include model (in-)validation, prediction, fault diagnosis, and controller design. We present an approach to determine inner- and outer-approximations of the set containing all consistent initial conditions/parameterizations for nonlinear continuous-time systems. These approximations are found by occupation measures that encode the system dynamics and measurements, and give rise to an infinite-dimensional linear program. We exploit the flexibility and linearity of the decision problem to incorporate uncertain-but-bounded and pointwise-in-time state and output constraints, a feature which was not addressed in previous works. The infinite-dimensional linear program is relaxed by a hierarchy of LMI problems that provide certificates in case no consistent initial condition/parameterization exists. Furthermore, the applied LMI relaxation guarantees that the approximations converge (almost uniformly) to the true consistent set. We illustrate the approach with a biochemical reaction network involving unknown initial conditions and parameters.

math.OC↗