arXiv subjects

Zlatko Drmač

Publications and source records attributed to Zlatko Drmač.

At least 19 recordsLinked to original sources

arXiv2026

On residual bounds of the EDMD solution to the eigenvalue problem for the Koopman operator and backward shadowing stability of the EDMD/KMD

This paper introduces shadowing theory based backward stability assessment of data driven computational analysis of discrete dynamical systems in the framework of the Koopman composition operator and the Extended Dynamic Mode Decomposition. Data driven spectral analysis of the dynamics using the approximate (computed) eigenpairs of the Koopman operator is cast in terms of the backward error analysis. The individual residuals of the computed eigenpairs are aggregated in a backward perturbation of the Koopman operator, so that the computed eigenpairs are exact for a perturbed operator that is not a composition operator. Then, the impact of the perturbation in the operator is carried over to the map and the initial condition of the original system, showing that the computed approximations correspond exactly to a pseudo--trajectory of the system. In the final step, the pseudo--trajectory is shadowed by an exact trajectory of the system. This interpretation of errors is in particular suitable in data driven scenarios where the data is contaminated by noise. New insights into the numerical shadowing are provided.

math.NA↗

arXiv2026

New Robust Streaming DMD with Forecasting

The Dynamic Mode Decomposition (DMD) and the more general Extended DMD (EDMD) are powerful tools for computational analysis of dynamical systems in data-driven scenarios. They are built on the theoretical foundation of the Koopman composition operator and can be considered as numerical methods for data snapshot-based extraction of spectral information of the composition operator associated with the dynamics, spectral analysis of the structure of the dynamics, and for forecasting. In high fidelity numerical simulations, the state space is high dimensional and efficient numerical methods leverage the fact that the actual dynamics evolves on manifolds of much smaller dimension. This motivates computing low rank approximations in a streaming fashion and the DMD matrix is adaptively updated with newly received data. In this way, large number of high dimensional snapshots can be processed very efficiently. Low dimensional representation also requires fast updating for online applications. This paper revisits the pioneering works of Hemati, Williams and Rowley (Physics of Fluids, 2014), and Zhang, Rowley, Deem and Cattafesta (SIAM Journal on Applied Dynamical Systems, 2019) on the streaming DMD and proposes improvements in functionality (using residual bounds, Exact DMD vectors), computational efficiency (more efficient algorithm with smaller memory footprint) and numerical robustness (smaller condition numbers and better forecasting skill).

math.NA↗

arXiv2025

An Introductory Guide to Koopman Learning

Koopman operators provide a linear framework for data-driven analyses of nonlinear dynamical systems, but their infinite-dimensional nature presents major computational challenges. In this article, we offer an introductory guide to Koopman learning, emphasizing rigorously convergent data-driven methods for forecasting and spectral analysis. We provide a unified account of error control via residuals in both finite- and infinite-dimensional settings, an elementary proof of convergence for generalized Laplace analysis -- a variant of filtered power iteration that works for operators with continuous spectra and no spectral gaps -- and review state-of-the-art approaches for computing continuous spectra and spectral measures. The goal is to provide both newcomers and experts with a clear, structured overview of reliable data-driven techniques for Koopman spectral analysis.

math.NA↗

arXiv2024

A data driven Koopman-Schur decomposition for computational analysis of nonlinear dynamics

This paper introduces a new theoretical and computational framework for a data driven Koopman mode analysis of nonlinear dynamics. To alleviate the potential problem of ill-conditioned eigenvectors in the existing implementations of the Dynamic Mode Decomposition (DMD) and the Extended Dynamic Mode Decomposition (EDMD), the new method introduces a Koopman-Schur decomposition that is entirely based on unitary transformations. The analysis in terms of the eigenvectors as modes of a Koopman operator compression is replaced with a modal decomposition in terms of a flag of invariant subspaces that correspond to selected eigenvalues. The main computational tool from the numerical linear algebra is the partial ordered Schur decomposition that provides convenient orthonormal bases for these subspaces. In the case of real data, a real Schur form is used and the computation is based on real orthogonal transformations. The new computational scheme is presented in the framework of the Extended DMD and the kernel trick is used.

math.NA↗

arXiv2024

IRKA is a Riemannian Gradient Descent Method

The iterative rational Krylov algorithm (IRKA) is a commonly used fixed-point iteration developed to minimize the $\mathcal{H}_2$ model order reduction error. In this work, IRKA is recast as a Riemannian gradient descent method with a fixed step size over the manifold of rational functions having fixed degree. This interpretation motivates the development of a Riemannian gradient descent method utilizing as a natural extension variable step size and line search. Comparisons made between IRKA and this extension on a few examples demonstrate significant benefits.

math.NA↗

arXiv2022

Numerical considerations and a new implementation for ICS

Invariant Coordinate Selection (ICS) is a multivariate data transformation and a dimension reduction method that can be useful in many different contexts. It can be used for outlier detection or cluster identification, and can be seen as an independent component or a non-Gaussian component analysis method. The usual implementation of ICS is based on a joint diagonalization of two scatter matrices, and may be numerically unstable in some ill-conditioned situations. We focus on one-step M-scatter matrices and propose a new implementation of ICS based on a pivoted QR factorization of the centered data set. This factorization avoids the direct computation of the scatter matrices and their inverse and brings numerical stability to the algorithm. Furthermore, the row and column pivoting leads to a rank revealing procedure that allows computation of ICS when the scatter matrices are not full rank. Several artificial and real data sets illustrate the interest of using the new implementation compared to the original one.

stat.CO↗

arXiv2021

An algorithm for the complete solution of the quartic eigenvalue problem

Quartic eigenvalue problem $(λ^4 A + λ^3 B + λ^2C + λD + E)x = \mathbf{0}$ naturally arises e.g. when solving the Orr-Sommerfeld equation in the analysis of the stability of the {Poiseuille} flow, in theoretical analysis and experimental design of locally resonant phononic plates, modeling a robot with electric motors in the joints, calibration of catadioptric vision system, or e.g. computation of the guided and leaky modes of a planar waveguide. This paper proposes a new numerical method for the full solution (all eigenvalues and all left and right eigenvectors) that is based on quadratification, i.e. reduction of the quartic problem to a spectraly equivalent quadratic eigenvalue problem, and on a careful preprocessing to identify and deflate zero and infinite eigenvalues before the linearized quadratification is forwarded to the QZ algorithm. Numerical examples and backward error analysis confirm that the proposed algorithm is superior to the available methods.

math.NA↗

arXiv2020

Predicting the Critical Number of Layers for Hierarchical Support Vector Regression

Hierarchical support vector regression (HSVR) models a function from data as a linear combination of SVR models at a range of scales, starting at a coarse scale and moving to finer scales as the hierarchy continues. In the original formulation of HSVR, there were no rules for choosing the depth of the model. In this paper, we observe in a number of models a phase transition in the training error -- the error remains relatively constant as layers are added, until a critical scale is passed, at which point the training error drops close to zero and remains nearly constant for added layers. We introduce a method to predict this critical scale a priori with the prediction based on the support of either a Fourier transform of the data or the Dynamic Mode Decomposition (DMD) spectrum. This allows us to determine the required number of layers prior to training any models.

cs.LG↗

arXiv2020

Learning low-dimensional dynamical-system models from noisy frequency-response data with Loewner rational interpolation

Loewner rational interpolation provides a versatile tool to learn low-dimensional dynamical-system models from frequency-response measurements. This work investigates the robustness of the Loewner approach to noise. The key finding is that if the measurements are polluted with Gaussian noise, then the error due to noise grows at most linearly with the standard deviation with high probability under certain conditions. The analysis gives insights into making the Loewner approach robust against noise via linear transformations and judicious selections of measurements. Numerical results demonstrate the linear growth of the error on benchmark examples.

math.NA↗

arXiv2020

Applications of Koopman Mode Analysis to Neural Networks

We consider the training process of a neural network as a dynamical system acting on the high-dimensional weight space. Each epoch is an application of the map induced by the optimization algorithm and the loss function. Using this induced map, we can apply observables on the weight space and measure their evolution. The evolution of the observables are given by the Koopman operator associated with the induced dynamical system. We use the spectrum and modes of the Koopman operator to realize the above objectives. Our methods can help to, a priori, determine the network depth; determine if we have a bad initialization of the network weights, allowing a restart before training too long; speeding up the training time. Additionally, our methods help enable noise rejection and improve robustness. We show how the Koopman spectrum can be used to determine the number of layers required for the architecture. Additionally, we show how we can elucidate the convergence versus non-convergence of the training process by monitoring the spectrum, in particular, how the existence of eigenvalues clustering around 1 determines when to terminate the learning process. We also show how using Koopman modes we can selectively prune the network to speed up the training procedure. Finally, we show that incorporating loss functions based on negative Sobolev norms can allow for the reconstruction of a multi-scale signal polluted by very large amounts of noise.

cs.LG↗

arXiv2020

Numerical methods for accurate computation of the eigenvalues of Hermitian matrices and the singular values of general matrices

This paper offers a review of numerical methods for computation of the eigenvalues of Hermitian matrices and the singular values of general and some classes of structured matrices. The focus is on the main principles behind the methods that guarantee high accuracy even in the cases that are ill-conditioned for the conventional methods. First, it is shown that a particular structure of the errors in a finite precision implementation of an algorithm allows for a much better measure of sensitivity and that computation with high accuracy is possible despite a large classical condition number. Such structured errors incurred by finite precision computation are in some algorithms e.g. entry-wise or column-wise small, which is much better than the usually considered errors that are in general small only when measured in the Frobenius matrix norm. Specially tailored perturbation theory for such structured perturbations of Hermitian matrices guarantees much better bounds for the relative errors in the computed eigenvalues. % Secondly, we review an unconventional approach to accurate computation of the singular values and eigenvalues of some notoriously ill-conditioned structured matrices, such as e.g. Cauchy, Vandermonde and Hankel matrices. The distinctive feature of accurate algorithms is using the intrinsic parameters that define such matrices to obtain a non-orthogonal factorization, such as the \textsf{LDU} factorization, and then computing the singular values of the product of thus computed factors. The state of the art software is discussed as well.

math.NA↗

arXiv2020

Stability of discrete empirical interpolation and gappy proper orthogonal decomposition with randomized and deterministic sampling points

This work investigates the stability of (discrete) empirical interpolation for nonlinear model reduction and state field approximation from measurements. Empirical interpolation derives approximations from a few samples (measurements) via interpolation in low-dimensional spaces. It has been observed that empirical interpolation can become unstable if the samples are perturbed due to, e.g., noise, turbulence, and numerical inaccuracies. The main contribution of this work is a probabilistic analysis that shows that stable approximations are obtained if samples are randomized and if more samples than dimensions of the low-dimensional spaces are used. Oversampling, i.e., taking more sampling points than dimensions of the low-dimensional spaces, leads to approximations via regression and is known under the name of gappy proper orthogonal decomposition. Building on the insights of the probabilistic analysis, a deterministic sampling strategy is presented that aims to achieve lower approximation errors with fewer points than randomized sampling by taking information about the low-dimensional spaces into account. Numerical results of reconstructing velocity fields from noisy measurements of combustion processes and model reduction in the presence of noise demonstrate the instability of empirical interpolation and the stability of gappy proper orthogonal decomposition with oversampling.

math.NA↗

arXiv2019

New robust ScaLAPACK routine for computing the QR factorization with column pivoting

In this note we describe two modifications of the ScaLAPACK subroutines PxGEQPF for computing the QR factorization with the Businger-Golub column pivoting. First, we resolve a subtle numerical instability in the same way as we have done it for the LAPACK subroutines xGEQPF, xGEQP3 in 2006. [LAPACK Working Note 176 (2006); ACM Trans. Math. Softw. 2008]. The problem originates in the first release of LINPACK in the 1970's: due to severe cancellations in the down-dating of partial column norms, the pivoting procedure may be in the dark completely about the true norms of the pivot column candidates. This may cause miss-pivoting, and as a result loss of the important rank revealing structure of the computed triangular factor, with severe consequences on other solvers that rely on the rank revealing pivoting. The instability is so subtle that, e.g., inserting a WRITE statement or changing the process topology can drastically change the result. Secondly, we also correct a programming error in the complex subroutines PCGEQPF, PZGEQPF, which also causes wrong pivoting because of erroneous use of PSCNRM2, PDZNRM2 for the explicit norm computation.

cs.MS↗

arXiv2019

New numerical algorithm for deflation of infinite and zero eigenvalues and full solution of quadratic eigenvalue problems

This paper presents a new method for computing all eigenvalues and eigenvectors of quadratic matrix pencil. It is an upgrade of the quadeig algorithm by Hammarling, Munro and Tisseur, which attempts to reveal and remove by deflation certain number of zero and infinite eigenvalues before QZ iterations. Proposed modifications of the quadeig framework are designed to enhance backward stability and to make the process of deflating infinite and zero eigenvalues more numerically robust. In particular, careful preprocessing allows scaling invariant/component-wise backward error and thus better condition number. Further, using an upper triangular version of the Kronecker canonical form enables deflating additional infinite eigenvalues, in addition to those inferred from the rank of leading coefficient matrix. Theoretical analysis and empirical evidence from thorough testing of the software implementation confirm superior numerical performances of the proposed method.

math.NA↗

arXiv2018

On least squares problems with certain Vandermonde--Khatri--Rao structure with applications to DMD

This paper proposes a new computational method for solving structured least squares problems that arise in the process of identification of coherent structures in fluid flows. It is deployed in combination with dynamic mode decomposition (DMD) which provides a non-orthogonal set of modes --- corresponding to particular temporal frequencies --- a subset of which is used to represent time snapshots of the underlying dynamics. The coefficients of the representation are determined from a solution of a structured linear least squares problem with the matrix that involves the Khatri--Rao product of a triangular and a Vandermonde matrix. Such a structure allows a very efficient normal equation based least squares solution, which is used in state of the art CFD tools such as the sparsity promoting DMD (DMDSP). A new numerical analysis of the normal equations approach provides insights about its applicability and its limitations. Relevant condition numbers that determine numerical robustness are identified and discussed. Further, the paper offers a corrected semi-normal solution and QR factorization based algorithms. It is shown how to use the Vandermonde--Khatri--Rao structure to efficiently compute the QR factorization of the least squares coefficient matrix, thus providing a new computational tool for the ill-conditioned cases where the normal equations may fail to compute a sufficiently accurate solution. Altogether, the presented material provides a firm numerical linear algebra framework for a class of structured least squares problems arising in a variety of applications.

math.NA↗

arXiv2018

Data driven Koopman spectral analysis in Vandermonde-Cauchy form via the DFT: numerical method and theoretical insights

The goals and contributions of this paper are twofold. It provides a new computational tool for data driven Koopman spectral analysis by taking up the formidable challenge to develop a numerically robust algorithm by following the natural formulation via the Krylov decomposition with the Frobenius companion matrix, and by using its eigenvectors explicitly -- these are defined as the inverse of the notoriously ill-conditioned Vandermonde matrix. The key step to curb ill-conditioning is the discrete Fourier transform of the snapshots; in the new representation, the Vandermonde matrix is transformed into a generalized Cauchy matrix, which then allows accurate computation by specially tailored algorithms of numerical linear algebra. The second goal is to shed light on the connection between the formulas for optimal reconstruction weights when reconstructing snapshots using subsets of the computed Koopman modes. It is shown how using a certain weaker form of generalized inverses leads to explicit reconstruction formulas that match the abstract results from Koopman spectral theory, in particular the Generalized Laplace Analysis.

math.NA↗

arXiv2018

Empirical least-squares fitting of parametrized dynamical systems

Given a set of response observations for a parametrized dynamical system, we seek a parametrized dynamical model that will yield uniformly small response error over a range of parameter values yet has low order. Frequently, access to internal system dynamics or equivalently, to realizations of the original system is either not possible or not practical; only response observations over a range of parameter settings might be known. Respecting these typical operational constraints, we propose a two phase approach that first encodes the response data into a high fidelity intermediate model of modest order, followed then by a compression stage that serves to eliminate redundancy in the intermediate model. For the first phase, we extend non-parametric least-squares fitting approaches so as to accommodate parameterized systems. This results in a (discrete) least-squares problem formulated with respect to both frequency and parameter that identifies "local" system response features. The second phase uses an $\mathbf{\mathcal{H}}_2$-optimal model reduction strategy accommodating the specialized parametric structure of the intermediate model obtained in the first phase. The final compressed model inherits the parametric dependence of the intermediate model and maintains the high fidelity of the intermediate model, while generally having dramatically smaller system order. We provide a variety of numerical examples demonstrating our approach.

math.NA↗

arXiv2018

The Discrete Empirical Interpolation Method: Canonical Structure and Formulation in Weighted Inner Product Spaces

New contributions are offered to the theory and practice of the Discrete Empirical Interpolation Method (DEIM). These include a detailed characterization of the canonical structure; a substantial tightening of the error bound for the DEIM oblique projection, based on index selection via a strong rank revealing QR factorization; and an extension of the DEIM approximation to weighted inner products defined by a real symmetric positive-definite matrix $W$. The weighted DEIM ($W$-DEIM) can be deployed in the more general framework where the POD Galerkin projection is formulated in a discretization of a suitable energy inner product such that the Galerkin projection preserves important physical properties such as e.g. stability. Also, a special case of $W$-DEIM is introduced, which is DGEIM, a discrete version of the Generalized Empirical Interpolation Method that allows generalization of the interpolation via a dictionary of linear functionals.

math.NA↗