arXiv subjects

Maarten Schoukens

Publications and source records attributed to Maarten Schoukens.

At least 55 records · Page 3Linked to original sources

arXiv2021

Non-linear State-space Model Identification from Video Data using Deep Encoders

Identifying systems with high-dimensional inputs and outputs, such as systems measured by video streams, is a challenging problem with numerous applications in robotics, autonomous vehicles and medical imaging. In this paper, we propose a novel non-linear state-space identification method starting from high-dimensional input and output data. Multiple computational and conceptual advances are combined to handle the high-dimensional nature of the data. An encoder function, represented by a neural network, is introduced to learn a reconstructability map to estimate the model states from past inputs and outputs. This encoder function is jointly learned with the dynamics. Furthermore, multiple computational improvements, such as an improved reformulation of multiple shooting and batch optimization, are proposed to keep the computational time under control when dealing with high-dimensional and large datasets. We apply the proposed method to a video stream of a simulated environment of a controllable ball in a unit box. The study shows low simulation error with excellent long term prediction capability of the model obtained using the proposed method.

eess.SY↗

arXiv2021

Nonlinear state-space identification using deep encoder networks

Nonlinear state-space identification for dynamical systems is most often performed by minimizing the simulation error to reduce the effect of model errors. This optimization problem becomes computationally expensive for large datasets. Moreover, the problem is also strongly non-convex, often leading to sub-optimal parameter estimates. This paper introduces a method that approximates the simulation loss by splitting the data set into multiple independent sections similar to the multiple shooting method. This splitting operation allows for the use of stochastic gradient optimization methods which scale well with data set size and has a smoothing effect on the non-convex cost function. The main contribution of this paper is the introduction of an encoder function to estimate the initial state at the start of each section. The encoder function estimates the initial states using a feed-forward neural network starting from historical input and output samples. The efficiency and performance of the proposed state-space encoder method is illustrated on two well-known benchmarks where, for instance, the method achieves the lowest known simulation error on the Wiener--Hammerstein benchmark.

cs.LG↗

arXiv2021

Improved Initialization of State-Space Artificial Neural Networks

The identification of black-box nonlinear state-space models requires a flexible representation of the state and output equation. Artificial neural networks have proven to provide such a representation. However, as in many identification problems, a nonlinear optimization problem needs to be solved to obtain the model parameters (layer weights and biases). A well-thought initialization of these model parameters can often avoid that the nonlinear optimization algorithm converges to a poorly performing local minimum of the considered cost function. This paper introduces an improved initialization approach for nonlinear state-space models represented as a recurrent artificial neural network and emphasizes the importance of including an explicit linear term in the model structure. Some of the neural network weights are initialized starting from a linear approximation of the nonlinear system, while others are initialized using random values or zeros. The effectiveness of the proposed initialization approach over previously proposed methods is illustrated on two benchmark examples.

cs.LG↗

arXiv2020

Toolbox for Discovering Dynamic System Relations via TAG Guided Genetic Programming

Data-driven modeling of nonlinear dynamical systems often require an expert user to take critical decisions a priori to the identification procedure. Recently an automated strategy for data driven modeling of \textit{single-input single-output} (SISO) nonlinear dynamical systems based on \textit{Genetic Programming} (GP) and \textit{Tree Adjoining Grammars} (TAG) has been introduced. The current paper extends these latest findings by proposing a \textit{multi-input multi-output} (MIMO) TAG modeling framework for polynomial NARMAX models. Moreover we introduce a TAG identification toolbox in Matlab that provides implementation of the proposed methodology to solve multi-input multi-output identification problems under NARMAX noise assumption. The capabilities of the toolbox and the modelling methodology are demonstrated in the identification of two SISO and one MIMO nonlinear dynamical benchmark models.

eess.SY↗

arXiv2020

Feedback Identification of conductance-based models

This paper applies the classical prediction error method (PEM) to the estimation of nonlinear discrete-time models of neuronal systems subject to input-additive noise. While the nonlinear system exhibits excitability, bifurcations, and limit-cycle oscillations, we prove consistency of the parameter estimation procedure under output feedback. Hence, this paper provides a rigorous framework for the application of conventional nonlinear system identification methods to discrete-time stochastic neuronal systems. The main result exploits the elementary property that conductance-based models of neurons have an exponentially contracting inverse dynamics. This property is implied by the voltage-clamp experiment, which has been the fundamental modeling experiment of neurons ever since the pioneering work of Hodgkin and Huxley.

eess.SY↗

arXiv2020

System identification of biophysical neuronal models

After sixty years of quantitative biophysical modeling of neurons, the identification of neuronal dynamics from input-output data remains a challenging problem, primarily due to the inherently nonlinear nature of excitable behaviors. By reformulating the problem in terms of the identification of an operator with fading memory, we explore a simple approach based on a parametrization given by a series interconnection of Generalized Orthonormal Basis Functions (GOBFs) and static Artificial Neural Networks. We show that GOBFs are particularly well-suited to tackle the identification problem, and provide a heuristic for selecting GOBF poles which addresses the ultra-sensitivity of neuronal behaviors. The method is illustrated on the identification of a bursting model from the crab stomatogastric ganglion.

q-bio.NC↗

arXiv2020

A Tree Adjoining Grammar Representation for Models Of Stochastic Dynamical Systems

Model structure and complexity selection remains a challenging problem in system identification, especially for parametric non-linear models. Many Evolutionary Algorithm (EA) based methods have been proposed in the literature for estimating model structure and complexity. In most cases, the proposed methods are devised for estimating structure and complexity within a specified model class and hence these methods do not extend to other model structures without significant changes. In this paper, we propose a Tree Adjoining Grammar (TAG) for stochastic parametric models. TAGs can be used to generate models in an EA framework while imposing desirable structural constraints and incorporating prior knowledge. In this paper, we propose a TAG that can systematically generate models ranging from FIRs to polynomial NARMAX models. Furthermore, we demonstrate that TAGs can be easily extended to more general model classes, such as the non-linear Box-Jenkins model class, enabling the realization of flexible and automatic model structure and complexity selection via EA.

eess.SY↗

arXiv2020

On the Initialization of Nonlinear LFR Model Identification with the Best Linear Approximation

Balancing the model complexity and the representation capability towards the process to be captured remains one of the main challenges in nonlinear system identification. One possibility to reduce model complexity is to impose structure on the model representation. To this end, this work considers the linear fractional representation framework. In a linear fractional representation the linear dynamics and the system nonlinearities are modeled by two separate blocks that are interconnected with one another. This results in a structured, yet flexible model structure. Estimating such a model directly from input-output data is not a trivial task as the involved optimization is nonlinear in nature. This paper proposes an initialization scheme for the model parameters based on the best linear approximation of the system and shows that this approach results in high quality models on a set of benchmark data sets.

eess.SY↗

arXiv2020

Best Linear Approximation of Nonlinear Continuous-Time Systems Subject to Process Noise and Operating in Feedback

In many engineering applications the level of nonlinear distortions in frequency response function (FRF) measurements is quantified using specially designed periodic excitation signals called random phase multisines and periodic noise. The technique is based on the concept of the best linear approximation (BLA) and it allows one to check the validity of the linear framework with a simple experiment. Although the classical BLA theory can handle measurement noise only, in most applications the noise generated by the system -- called process noise -- is the dominant noise source. Therefore, there is a need to extend the existing BLA theory to the process noise case. In this paper we study in detail the impact of the process noise on the BLA of nonlinear continuous-time systems operating in a closed loop. It is shown that the existing nonparametric estimation methods for detecting and quantifying the level of nonlinear distortions in FRF measurements are still applicable in the presence of process noise. All results are also valid for discrete-time systems and systems operating in open loop.

eess.SY↗

arXiv2020

Feedback for nonlinear system identification

Motivated by neuronal models from neuroscience, we consider the system identification of simple feedback structures whose behaviors include nonlinear phenomena such as excitability, limit-cycles and chaos. We show that output feedback is sufficient to solve the identification problem in a two-step procedure. First, the nonlinear static characteristic of the system is extracted, and second, using a feedback linearizing law, a mildly nonlinear system with an approximately-finite memory is identified. In an ideal setting, the second step boils down to the identification of a LTI system. To illustrate the method in a realistic setting, we present numerical simulations of the identification of two classical systems that fit the assumed model structure.

eess.SY↗

arXiv2019

Data-driven Modelling of Dynamical Systems Using Tree Adjoining Grammar and Genetic Programming

State-of-the-art methods for data-driven modelling of non-linear dynamical systems typically involve interactions with an expert user. In order to partially automate the process of modelling physical systems from data, many EA-based approaches have been proposed for model-structure selection, with special focus on non-linear systems. Recently, an approach for data-driven modelling of non-linear dynamical systems using Genetic Programming (GP) was proposed. The novelty of the method was the modelling of noise and the use of Tree Adjoining Grammar to shape the search-space explored by GP. In this paper, we report results achieved by the proposed method on three case studies. Each of the case studies considered here is based on real physical systems. The case studies pose a variety of challenges. In particular, these challenges range over varying amounts of prior knowledge of the true system, amount of data available, the complexity of the dynamics of the system, and the nature of non-linearities in the system. Based on the results achieved for the case studies, we critically analyse the performance of the proposed method.

eess.SY↗

arXiv2018

Grammar-based Representation and Identification of Dynamical Systems

In this paper we propose a novel approach to identify dynamical systems. The method estimates the model structure and the parameters of the model simultaneously, automating the critical decisions involved in identification such as model structure and complexity selection. In order to solve the combined model structure and model parameter estimation problem, a new representation of dynamical systems is proposed. The proposed representation is based on Tree Adjoining Grammar, a formalism that was developed from linguistic considerations. Using the proposed representation, the identification problem can be interpreted as a multi-objective optimization problem and we propose a Evolutionary Algorithm-based approach to solve the problem. A benchmark example is used to demonstrate the proposed approach. The results were found to be comparable to that obtained by state-of-the-art non-linear system identification methods, without making use of knowledge of the system description.

eess.SY↗

arXiv2018

On the Simulation of Polynomial NARMAX Models

In this paper, we show that the common approach for simulation non-linear stochastic models, commonly used in system identification, via setting the noise contributions to zero results in a biased response. We also demonstrate that to achieve unbiased simulation of finite order NARMAX models, in general, we require infinite order simulation models. The main contributions of the paper are two-fold. Firstly, an alternate representation of polynomial NARMAX models, based on Hermite polynomials, is proposed. The proposed representation provides a convenient way to translate a polynomial NARMAX model to a corresponding simulation model by simply setting certain terms to zero. This translation is exact when the simulation model can be written as an NFIR model. Secondly, a parameterized approximation method is proposed to curtail infinite order simulation models to a finite order. The proposed approximation can be viewed as a trade-off between the conventional approach of setting noise contributions to zero and the approach of incorporating the bias introduced by higher-order moments of the noise distribution. Simulation studies are provided to illustrate the utility of the proposed representation and approximation method.

eess.SY↗

arXiv2018

From Nonlinear Identification to Linear Parameter Varying Models: Benchmark Examples

Linear parameter-varying (LPV) models form a powerful model class to analyze and control a (nonlinear) system of interest. Identifying a LPV model of a nonlinear system can be challenging due to the difficulty of selecting the scheduling variable(s) a priori, which is quite challenging in case a first principles based understanding of the system is unavailable. This paper presents a systematic LPV embedding approach starting from nonlinear fractional representation models. A nonlinear system is identified first using a nonlinear block-oriented linear fractional representation (LFR) model. This nonlinear LFR model class is embedded into the LPV model class by factorization of the static nonlinear block present in the model. As a result of the factorization a LPV-LFR or a LPV state-space model with an affine dependency results. This approach facilitates the selection of the scheduling variable from a data-driven perspective. Furthermore the estimation is not affected by measurement noise on the scheduling variables, which is often left untreated by LPV model identification methods. The proposed approach is illustrated on two well-established nonlinear modeling benchmark examples.

eess.SY↗

arXiv2018

Linear Parameter Varying Representation of a class of MIMO Nonlinear Systems

Linear parameter-varying (LPV) models form a powerful model class to analyze and control a (nonlinear) system of interest. Identifying an LPV model of a nonlinear system can be challenging due to the difficulty of selecting the scheduling variable(s) a priori, especially if a first principles based understanding of the system is unavailable. Converting a nonlinear model to an LPV form is also non-trivial and requires systematic methods to automate the process. Inspired by these challenges, a systematic LPV embedding approach starting from multiple-input multiple-output (MIMO) linear fractional representations with a nonlinear feedback block (NLFR) is proposed. This NLFR model class is embedded into the LPV model class by an automated factorization of the (possibly MIMO) static nonlinear block present in the model. As a result of the factorization, an LPV-LFR or an LPV state-space model with affine dependency on the scheduling is obtained. This approach facilitates the selection of the scheduling variable and the connected mapping of system variables. Such a conversion method enables to use nonlinear identification tools to estimate LPV models. The potential of the proposed approach is illustrated on a 2-DOF nonlinear mass-spring-damper example.

eess.SY↗

arXiv2018

Structure detection of Wiener-Hammerstein systems with process noise

Identification of nonlinear block-oriented models has been extensively studied. The presence of the process noise, more precisely its location in the block-oriented model influences essentially the development of a consistent identification algorithm. The present work is proposed with the aim to localize the process noise in the block-oriented model for accurate nonlinear modeling. To this end, the response of a Wiener-Hammerstein system is theoretically analyzed, the disturbance component in the output, caused by the process noise preceding the static nonlinearity, is shown to be dependent on the input signal. Inspired by such theoretical observation, a simple and new protocol is developed to determine the location of the process noise with respect to the static nonlinearity by using an input signal that is periodic, but nonstationary within one period. In addition, the proposed technique is promising to detect the type of certain static nonlinearity (e.g., dead-zone, saturation). Finally, it is validated on a simulated example and a real-life benchmark.

eess.SY↗

arXiv2018

Structure Discrimination in Block-Oriented Models Using Linear Approximations: A Theoretic Framework

In this paper we show that it is possible to retrieve structural information about complex block-oriented nonlinear systems, starting from linear approximations of the nonlinear system around different setpoints.The key idea is to monitor the movements of the poles and zeros of the linearized models and to reduce the number of candidate models on the basis of these observations. Besides the well known open loop single branch Wiener-, Hammerstein-, and Wiener-Hammerstein systems, we also cover a number of more general structures like parallel (multi branch) Wiener-Hammerstein models, and closed loop block oriented models, including linear fractional representation (LFR) models.

eess.SY↗

arXiv2018

Extending the Best Linear Approximation Framework to the Process Noise Case

The Best Linear Approximation (BLA) framework has already proven to be a valuable tool to analyze nonlinear systems and to start the nonlinear modeling process. The existing BLA framework is limited to systems with additive (colored) noise at the output. Such a noise framework is a simplified representation of reality. Process noise can play an important role in many real-life applications. This paper generalizes the Best Linear Approximation framework to account also for the process noise, both for the open-loop and the closed-loop setting, and shows that the most important properties of the existing BLA framework remain valid. The impact of the process noise contributions on the robust BLA estimation method is also analyzed.

eess.SY↗