arXiv subjects

Mingzhou Yin

Publications and source records attributed to Mingzhou Yin.

At least 19 recordsLinked to original sources

arXiv2025

A Unified Bayesian Framework for Data-Driven Smoothing, Prediction, and Control

Extending data-driven algorithms based on Willems' fundamental lemma to stochastic data often requires empirical and customized workarounds. This work presents a unified Bayesian framework for linear systems that provides a systematic and general method for handling stochastic data-driven tasks, including smoothing, prediction, and control, via maximum a posteriori estimation. This framework formulates a unified trajectory estimation problem for the three tasks by specifying different types of trajectory knowledge. Then, a Bayesian problem is solved that optimally combines trajectory knowledge with a data-driven characterization of the trajectory from offline data for correlated input-output uncertainties with elliptical distributions. Under specific conditions, this problem is shown to generalize existing data-driven prediction and control algorithms. Numerical examples demonstrate the performance of the unified approach for all three tasks against other data-driven and system identification approaches.

eess.SY↗

arXiv2025

Low-Rank Matrix Regression via Least-Angle Regression

Low-rank matrix regression is a fundamental problem in data science with various applications in systems and control. Nuclear norm regularization has been widely applied to solve this problem due to its convexity. However, it suffers from high computational complexity and the inability to directly specify the rank. This work introduces a novel framework for low-rank matrix regression that addresses both unstructured and Hankel matrices. By decomposing the low-rank matrix into rank-1 bases, the problem is reformulated as an infinite-dimensional sparse learning problem. The least-angle regression (LAR) algorithm is then employed to solve this problem efficiently. For unstructured matrices, a closed-form LAR solution is derived with equivalence to a normalized nuclear norm regularization problem. For Hankel matrices, a real-valued polynomial basis reformulation enables effective LAR implementation. Two numerical examples in network modeling and system realization demonstrate that the proposed approach significantly outperforms the nuclear norm method in terms of estimation accuracy and computational efficiency.

eess.SY↗

arXiv2025

Data-Driven Prediction and Control of Hammerstein-Wiener Systems with Implicit Gaussian Processes

This work investigates data-driven prediction and control of Hammerstein-Wiener systems using physics-informed Gaussian process (GP) models that encode the block-oriented model structure. Data-driven prediction algorithms have been developed for structured nonlinear systems based on Willems' fundamental lemma. However, existing frameworks do not apply to output nonlinearities in Wiener systems and rely on a finite-dimensional dictionary of basis functions for Hammerstein systems. In this work, an implicit predictor structure is considered, leveraging the linearity for the dynamical part of the model. This implicit function is learned by GP regression, utilizing carefully designed structured kernel functions from linear model parameters and GP priors for the nonlinearities. Virtual derivative points are added to the regression by expectation propagation to encode monotonicity information of the nonlinearities. The linear model parameters are estimated as hyperparameters by assuming a stable spline hyperprior. The implicit GP model provides explicit output prediction by optimizing selected optimality criteria. The implicit model is also applied to receding horizon control with the expected control cost and chance constraint satisfaction guarantee. Numerical results demonstrate that the proposed prediction and control algorithms are superior to black-box GP models without model structure knowledge.

eess.SY↗

arXiv2024

Optimal Data-Driven Prediction and Predictive Control using Signal Matrix Models

Data-driven control uses a past signal trajectory to characterise the input-output behaviour of a system. Willems' lemma provides a data-based prediction model allowing a control designer to bypass the step of identifying a state-space or transfer function model. This paper provides a more parsimonious formulation of Willems' lemma that separates the model into initial condition matching and predictive control design parts. This avoids the need for regularisers in the predictive control problem that are found in other data-driven predictive control methods. It also gives a closed form expression for the optimal (minimum variance) unbiased predictor of the future output trajectory and applies it for predictive control. Simulation comparisons illustrate very good control performance.

eess.SY↗

arXiv2023

Frequency-Domain Identification of Discrete-Time Systems using Sum-of-Rational Optimization

We propose a computationally tractable method for the identification of stable canonical discrete-time rational transfer function models, using frequency domain data. The problem is formulated as a global non-convex optimization problem whose objective function is the sum of weighted squared residuals at each observed frequency datapoint. Stability is enforced using a polynomial matrix inequality constraint. The problem is solved globally by a moment-sum-of-squares hierarchy of semidefinite programs through a framework for sum-of-rational-functions optimization. Convergence of the moment-sum-of-squares program is guaranteed as the bound on the degree of the sum-of-squares polynomials approaches infinity. The performance of the proposed method is demonstrated using numerical simulation examples.

eess.SY↗

arXiv2023

Stochastic Data-Driven Predictive Control: Regularization, Estimation, and Constraint Tightening

Data-driven predictive control methods based on the Willems' fundamental lemma have shown great success in recent years. These approaches use receding horizon predictive control with nonparametric data-driven predictors instead of model-based predictors. This study addresses three problems of applying such algorithms under unbounded stochastic uncertainties: 1) tuning-free regularizer design, 2) initial condition estimation, and 3) reliable constraint satisfaction, by using stochastic prediction error quantification. The regularizer is designed by leveraging the expected output cost. An initial condition estimator is proposed by filtering the measurements with the one-step-ahead stochastic data-driven prediction. A novel constraint-tightening method, using second-order cone constraints, is presented to ensure high-probability chance constraint satisfaction. Numerical results demonstrate that the proposed methods lead to satisfactory control performance in terms of both control cost and constraint satisfaction, with significantly improved initial condition estimation.

eess.SY↗

arXiv2023

Closed-Loop Identification of Stabilized Models Using Dual Input-Output Parameterization

This paper introduces a dual input-output parameterization (dual IOP) for the identification of linear time-invariant systems from closed-loop data. It draws inspiration from the recent input-output parameterization developed to synthesize a stabilizing controller. The controller is parameterized in terms of closed-loop transfer functions, from the external disturbances to the input and output of the system, constrained to lie in a given subspace. Analogously, the dual IOP method parameterizes the unknown plant with analogous closed-loop transfer functions, also referred to as dual parameters. In this case, these closed-loop transfer functions are constrained to lie in an affine subspace guaranteeing that the identified plant is \emph{stabilized} by the known controller. Compared with existing closed-loop identification techniques guaranteeing closed-loop stability, such as the dual Youla parameterization, the dual IOP neither requires a doubly-coprime factorization of the controller nor a nominal plant that is stabilized by the controller. The dual IOP does not depend on the order and the state-space realization of the controller either, as in the dual system-level parameterization. Simulation shows that the dual IOP outperforms the existing benchmark methods.

math.OC↗

arXiv2023

A Dual System-Level Parameterization for Identification from Closed-Loop Data

This work presents a dual system-level parameterization (D-SLP) method for closed-loop system identification. The recent system-level synthesis framework parameterizes all stabilizing controllers via linear constraints on closed-loop response functions, known as system-level parameters. It was demonstrated that several structural, locality, and communication constraints on the controller can be posed as convex constraints on these system-level parameters. In the current work, the identification problem is treated as a {\em dual} of the system-level synthesis problem. The plant model is identified from the dual system-level parameters associated to the plant. In comparison to existing closed-loop identification approaches (such as the dual-Youla parameterization), the D-SLP framework neither requires the knowledge of a nominal plant that is stabilized by the known controller, nor depends upon the choice of factorization of the nominal plant and the stabilizing controller. Numerical simulations demonstrate the efficacy of the proposed D-SLP method in terms of identification errors, compared to existing closed-loop identification techniques.

math.OC↗

arXiv2023

Error Bounds for Kernel-Based Linear System Identification with Unknown Hyperparameters

The kernel-based method has been successfully applied in linear system identification using stable kernel designs. From a Gaussian process perspective, it automatically provides probabilistic error bounds for the identified models from the posterior covariance, which are useful in robust and stochastic control. However, the error bounds require knowledge of the true hyperparameters in the kernel design and are demonstrated to be inaccurate with estimated hyperparameters for lightly damped systems or in the presence of high noise. In this work, we provide reliable quantification of the estimation error when the hyperparameters are unknown. The bounds are obtained by first constructing a high-probability set for the true hyperparameters from the marginal likelihood function and then finding the worst-case posterior covariance within the set. The proposed bound is proven to contain the true model with a high probability and its validity is verified in numerical simulation.

eess.SY↗

arXiv2022

Infinite-Dimensional Sparse Learning in Linear System Identification

Regularized methods have been widely applied to system identification problems without known model structures. This paper proposes an infinite-dimensional sparse learning algorithm based on atomic norm regularization. Atomic norm regularization decomposes the transfer function into first-order atomic models and solves a group lasso problem that selects a sparse set of poles and identifies the corresponding coefficients. The difficulty in solving the problem lies in the fact that there are an infinite number of possible atomic models. This work proposes a greedy algorithm that generates new candidate atomic models maximizing the violation of the optimality condition of the existing problem. This algorithm is able to solve the infinite-dimensional group lasso problem with high precision. The algorithm is further extended to reduce the bias and reject false positives in pole location estimation by iteratively reweighted adaptive group lasso and complementary pairs stability selection respectively. Numerical results demonstrate that the proposed algorithm performs better than benchmark parameterized and regularized methods in terms of both impulse response fitting and pole location estimation.

eess.SY↗

arXiv2022

Kernel-Based Identification of Local Limit Cycle Dynamics with Linear Periodically Parameter-Varying Models

Limit cycle oscillations are phenomena arising in nonlinear dynamical systems and characterized by periodic, locally-stable, and self-sustained state trajectories. Systems controlled in a closed loop along a periodic trajectory can also be modelled as systems experiencing limit cycle behavior. The goal of this work is to identify from data, the local dynamics around the limit cycle using linear periodically parameter-varying models. Using a coordinate transformation onto transversal surfaces, the dynamics are decomposed into two parts: one along the limit cycle, and one on the transversal surfaces. Then, the model is identified from trajectory data using kernel-based methods with a periodic kernel design. The kernel-based model is extended to also account for variations in system parameters associated with different operating conditions. The performance of the proposed identification method is demonstrated on a benchmark nonlinear system and on a simplified airborne wind energy model. The method provides accurate model parameter estimation, compared to the analytical linearization, and good prediction capability.

eess.SY↗

arXiv2021

Maximum Likelihood Estimation in Data-Driven Modeling and Control

Recently, various algorithms for data-driven simulation and control have been proposed based on the Willems' fundamental lemma. However, when collected data are noisy, these methods lead to ill-conditioned data-driven model structures. In this work, we present a maximum likelihood framework to obtain an optimal data-driven model, the signal matrix model, in the presence of output noise. Data compression and noise level estimation schemes are also proposed to apply the algorithm efficiently to large datasets and unknown noise level scenarios. Two approaches in system identification and receding horizon control are developed based on the derived optimal estimator. The first one identifies a finite impulse response model. This approach improves the least-squares estimator with less restrictive assumptions. The second one applies the signal matrix model as the predictor in predictive control. The control performance is shown to be better than existing data-driven predictive control algorithms, especially under high noise levels. Both approaches demonstrate that the derived estimator provides a promising framework to apply data-driven algorithms to noisy data.

eess.SY↗

arXiv2021

Maximum Likelihood Signal Matrix Model for Data-Driven Predictive Control

The paper presents a data-driven predictive control framework based on an implicit input-output mapping derived directly from the signal matrix of collected data. This signal matrix model is derived by maximum likelihood estimation with noise-corrupted data. By linearizing online, the implicit model can be used as a linear constraint to characterize possible trajectories of the system in receding horizon control. The signal matrix can also be updated online with new measurements. This algorithm can be applied to large datasets and slowly time-varying systems, possibly with high noise levels. An additional regularization term on the prediction error can be introduced to enhance the predictability and thus the control performance. Numerical results demonstrate that the proposed signal matrix model predictive control algorithm is effective in multiple applications and performs better than existing data-driven predictive control algorithm.

eess.SY↗

arXiv2021

Data-Driven Prediction with Stochastic Data: Confidence Regions and Minimum Mean-Squared Error Estimates

Recently, direct data-driven prediction has found important applications for controlling unknown systems, particularly in predictive control. Such an approach provides exact prediction using behavioral system theory when noise-free data are available. For stochastic data, although approximate predictors exist based on different statistical criteria, they fail to provide statistical guarantees of prediction accuracy. In this paper, confidence regions are provided for these stochastic predictors based on the prediction error distribution. Leveraging this, an optimal predictor which achieves minimum mean-squared prediction error is also proposed to enhance prediction accuracy. These results depend on some true model parameters, but they can also be replaced with an approximate data-driven formulation in practice. Numerical results show that the derived confidence region is valid and smaller prediction errors are observed for the proposed minimum mean-squared error estimate, even with the approximate data-driven formulation.

eess.SY↗

arXiv2021

Design of input for data-driven simulation with Hankel and Page matrices

The paper deals with the problem of designing informative input trajectories for data-driven simulation. First, the excitation requirements in the case of noise-free data are discussed and new weaker conditions, which assume the simulated input to be known in advance, are provided. Then, the case of noisy data trajectories is considered and an input design problem based on a recently proposed maximum likelihood estimator is formulated. A Bayesian interpretation is provided, and the implications of using Hankel and Page matrix representations are demonstrated. Numerical examples show the impact of the designed input on the predictive accuracy.

eess.SY↗

arXiv2021

Experiment design for impulse response identification with signal matrix models

This paper formulates an input design approach for truncated infinite impulse response identification in the context of implicit model representations recently used as basis for data-driven simulation and control approaches. Precisely, the considered model consists of a linear combination of the columns of a data (or signal) matrix. An optimal combination for the case of noisy data was recently proposed using a maximum likelihood approach, and the objective here is to optimize the input entries of the data matrix such that the mean-square error matrix of the estimate is minimized. A least-norm problem is derived in terms of the optimality criteria typically considered in the experiment design literature. Numerical results showcase the improved estimation fit achieved with the optimized input.

eess.SY↗

arXiv2020

On Low-Rank Hankel Matrix Denoising

The low-complexity assumption in linear systems can often be expressed as rank deficiency in data matrices with generalized Hankel structure. This makes it possible to denoise the data by estimating the underlying structured low-rank matrix. However, standard low-rank approximation approaches are not guaranteed to perform well in estimating the noise-free matrix. In this paper, recent results in matrix denoising by singular value shrinkage are reviewed. A novel approach is proposed to solve the low-rank Hankel matrix denoising problem by using an iterative algorithm in structured low-rank approximation modified with data-driven singular value shrinkage. It is shown numerically in both the input-output trajectory denoising and the impulse response denoising problems, that the proposed method performs the best in terms of estimating the noise-free matrix among existing algorithms of low-rank matrix approximation and denoising.

eess.SY↗

arXiv2020

Subspace Identification of Linear Time-Periodic Systems with Periodic Inputs

This paper proposes a new methodology for subspace identification of linear time-periodic (LTP) systems with periodic inputs. This method overcomes the issues related to the computation of frequency response of LTP systems by utilizing the frequency response of the time-lifted system with linear time-invariant structure instead. The response is estimated with an ensemble of input-output data with periodic inputs. This allows the frequency-domain subspace identification technique to be extended to LTP systems. The time-aliased periodic impulse response can then be estimated and the order-revealing decomposition of the block-Hankel matrix is formulated. The consistency of the proposed method is proved under mild noise assumptions. Numerical simulation shows that the proposed method performs better than multiple widely-used time-domain subspace identification methods when an ensemble of periodic data is available.

eess.SY↗