Physics-Informed Echo State Networks for Modeling Controllable Dynamical Systems

Abstract

Echo State Networks (ESNs) are recurrent neural networks usually employed for modeling nonlinear dynamic systems with relatively ease of training. By incorporating physical laws into the training of ESNs, Physics-Informed ESNs (PI-ESNs) were proposed initially to model chaotic dynamic systems without external inputs. They require less data for training since Ordinary Differential Equations (ODEs) of the considered system help to regularize the ESN. In this work, the PI-ESN is extended with external inputs to model controllable nonlinear dynamic systems. Additionally, an existing self-adaptive balancing loss method is employed to balance the contributions of the residual regression term and the physics-informed loss term in the total loss function. The experiments with two nonlinear systems modeled by ODEs, the Van der Pol oscillator and the four-tank system, and with one differential-algebraic (DAE) system, an electric submersible pump, revealed that the proposed PI-ESN outperforms the conventional ESN, especially in scenarios with limited data availability, showing that PI-ESNs can regularize an ESN model with external inputs previously trained on just a few datapoints, reducing its overfitting and improving its generalization error (up to 92% relative reduction in the test error). Further experiments demonstrated that the proposed PI-ESN is robust to parametric uncertainties in the ODE equations and that model predictive control using PI-ESN outperforms the one using plain ESN, particularly when training data is scarce.