arXiv subjects

Sui Tang

Publications and source records attributed to Sui Tang.

31 records · Page 2Linked to original sources

arXiv2022

Learning Transition Operators From Sparse Space-Time Samples

We consider the nonlinear inverse problem of learning a transition operator $\mathbf{A}$ from partial observations at different times, in particular from sparse observations of entries of its powers $\mathbf{A},\mathbf{A}^2,\cdots,\mathbf{A}^{T}$. This Spatio-Temporal Transition Operator Recovery problem is motivated by the recent interest in learning time-varying graph signals that are driven by graph operators depending on the underlying graph topology. We address the nonlinearity of the problem by embedding it into a higher-dimensional space of suitable block-Hankel matrices, where it becomes a low-rank matrix completion problem, even if $\mathbf{A}$ is of full rank. For both a uniform and an adaptive random space-time sampling model, we quantify the recoverability of the transition operator via suitable measures of incoherence of these block-Hankel embedding matrices. For graph transition operators these measures of incoherence depend on the interplay between the dynamics and the graph topology. We develop a suitable non-convex iterative reweighted least squares (IRLS) algorithm, establish its quadratic local convergence, and show that, in optimal scenarios, no more than $\mathcal{O}(rn \log(nT))$ space-time samples are sufficient to ensure accurate recovery of a rank-$r$ operator $\mathbf{A}$ of size $n \times n$. This establishes that spatial samples can be substituted by a comparable number of space-time samples. We provide an efficient implementation of the proposed IRLS algorithm with space complexity of order $O(r n T)$ and per-iteration time complexity linear in $n$. Numerical experiments for transition operators based on several graph models confirm that the theoretical findings accurately track empirical phase transitions, and illustrate the applicability and scalability of the proposed algorithm.

cs.IT↗

arXiv2021

Robust recovery of bandlimited graph signals via randomized dynamical sampling

Heat diffusion processes have found wide applications in modelling dynamical systems over graphs. In this paper, we consider the recovery of a $k$-bandlimited graph signal that is an initial signal of a heat diffusion process from its space-time samples. We propose three random space-time sampling regimes, termed dynamical sampling techniques, that consist in selecting a small subset of space-time nodes at random according to some probability distribution. We show that the number of space-time samples required to ensure stable recovery for each regime depends on a parameter called the spectral graph weighted coherence, that depends on the interplay between the dynamics over the graphs and sampling probability distributions. In optimal scenarios, no more than $\mathcal{O}(k \log(k))$ space-time samples are sufficient to ensure accurate and stable recovery of all $k$-bandlimited signals. In any case, dynamical sampling typically requires much fewer spatial samples than the static case by leveraging the temporal information. Then, we propose a computationally efficient method to reconstruct $k$-bandlimited signals from their space-time samples. We prove that it yields accurate reconstructions and that it is also stable to noise. Finally, we test dynamical sampling techniques on a wide variety of graphs. The numerical results support our theoretical findings and demonstrate the efficiency.

cs.IT↗

arXiv2021

Estimate the spectrum of affine dynamical systems from partial observations of a single trajectory data

In this paper, we study the nonlinear inverse problem of estimating the spectrum of a system matrix, that drives a finite-dimensional affine dynamical system, from partial observations of a single trajectory data. In the noiseless case, we prove an annihilating polynomial of the system matrix, whose roots are a subset of the spectrum, can be uniquely determined from data. We then study which eigenvalues of the system matrix can be recovered and derive various sufficient and necessary conditions to characterize the relationship between the recoverability of each eigenvalue and the observation locations. We propose various reconstruction algorithms, with theoretical guarantees, generalizing the classical Prony method, ESPIRIT, and matrix pencil method. We test the algorithms over a variety of examples with applications to graph signal processing, disease modeling and a real-human motion dataset. The numerical results validate our theoretical results and demonstrate the effectiveness of the proposed algorithms, even when the data did not follow an exact linear dynamical system.

math.NA↗

arXiv2020

Learning Theory for Inferring Interaction Kernels in Second-Order Interacting Agent Systems

Modeling the complex interactions of systems of particles or agents is a fundamental scientific and mathematical problem that is studied in diverse fields, ranging from physics and biology, to economics and machine learning. In this work, we describe a very general second-order, heterogeneous, multivariable, interacting agent model, with an environment, that encompasses a wide variety of known systems. We describe an inference framework that uses nonparametric regression and approximation theory based techniques to efficiently derive estimators of the interaction kernels which drive these dynamical systems. We develop a complete learning theory which establishes strong consistency and optimal nonparametric min-max rates of convergence for the estimators, as well as provably accurate predicted trajectories. The estimators exploit the structure of the equations in order to overcome the curse of dimensionality and we describe a fundamental coercivity condition on the inverse problem which ensures that the kernels can be learned and relates to the minimal singular value of the learning matrix. The numerical algorithm presented to build the estimators is parallelizable, performs well on high-dimensional problems, and is demonstrated on complex dynamical systems.

stat.ML↗

arXiv2020

On the identifiability of interaction functions in systems of interacting particles

We address a fundamental issue in the nonparametric inference for systems of interacting particles: the identifiability of the interaction functions. We prove that the interaction functions are identifiable for a class of first-order stochastic systems, including linear systems with general initial laws and nonlinear systems with stationary distributions. We show that a coercivity condition is sufficient for identifiability and becomes necessary when the number of particles approaches infinity. The coercivity is equivalent to the strict positivity of related integral operators, which we prove by showing that their integral kernels are strictly positive definite by using Müntz type theorems.

math.ST↗

arXiv2020

Learning interaction kernels in stochastic systems of interacting particles from multiple trajectories

We consider stochastic systems of interacting particles or agents, with dynamics determined by an interaction kernel which only depends on pairwise distances. We study the problem of inferring this interaction kernel from observations of the positions of the particles, in either continuous or discrete time, along multiple independent trajectories. We introduce a nonparametric inference approach to this inverse problem, based on a regularized maximum likelihood estimator constrained to suitable hypothesis spaces adaptive to data. We show that a coercivity condition enables us to control the condition number of this problem and prove the consistency of our estimator, and that in fact it converges at a near-optimal learning rate, equal to the min-max rate of $1$-dimensional non-parametric regression. In particular, this rate is independent of the dimension of the state space, which is typically very high. We also analyze the discretization errors in the case of discrete-time observations, showing that it is of order $1/2$ in terms of the time gaps between observations. This term, when large, dominates the sampling error and the approximation error, preventing convergence of the estimator. Finally, we exhibit an efficient parallel algorithm to construct the estimator from data, and we demonstrate the effectiveness of our algorithm with numerical tests on prototype systems including stochastic opinion dynamics and a Lennard-Jones model.

math.ST↗

arXiv2020

Learning interaction kernels in heterogeneous systems of agents from multiple trajectories

Systems of interacting particles or agents have wide applications in many disciplines such as Physics, Chemistry, Biology and Economics. These systems are governed by interaction laws, which are often unknown: estimating them from observation data is a fundamental task that can provide meaningful insights and accurate predictions of the behaviour of the agents. In this paper, we consider the inverse problem of learning interaction laws given data from multiple trajectories, in a nonparametric fashion, when the interaction kernels depend on pairwise distances. We establish a condition for learnability of interaction kernels, and construct estimators that are guaranteed to converge in a suitable $L^2$ space, at the optimal min-max rate for 1-dimensional nonparametric regression. We propose an efficient learning algorithm based on least squares, which can be implemented in parallel for multiple trajectories and is therefore well-suited for the high dimensional, big data regime. Numerical simulations on a variety examples, including opinion dynamics, predator-swarm dynamics and heterogeneous particle dynamics, suggest that the learnability condition is satisfied in models used in practice, and the rate of convergence of our estimator is consistent with the theory. These simulations also suggest that our estimators are robust to noise in the observations, and produce accurate predictions of dynamics in relative large time intervals, even when they are learned from data collected in short time intervals.

stat.ML↗

arXiv2019

Nonparametric inference of interaction laws in systems of agents from trajectory data

Inferring the laws of interaction between particles and agents in complex dynamical systems from observational data is a fundamental challenge in a wide variety of disciplines. We propose a non-parametric statistical learning approach to estimate the governing laws of distance-based interactions, with no reference or assumption about their analytical form, from data consisting trajectories of interacting agents. We demonstrate the effectiveness of our learning approach both by providing theoretical guarantees, and by testing the approach on a variety of prototypical systems in various disciplines. These systems include homogeneous and heterogeneous agents systems, ranging from particle systems in fundamental physics to agent-based systems modeling opinion dynamics under the social influence, prey-predator dynamics, flocking and swarming, and phototaxis in cell dynamics.

cs.LG↗

arXiv2019

Sensor Calibration for Off-the-Grid Spectral Estimation

This paper studies sensor calibration in spectral estimation where the true frequencies are located on a continuous domain. We consider a uniform array of sensors that collects measurements whose spectrum is composed of a finite number of frequencies, where each sensor has an unknown calibration parameter. Our goal is to recover the spectrum and the calibration parameters simultaneously from multiple snapshots of the measurements. In the noiseless case with an infinite number of snapshots, we prove uniqueness of this problem up to certain trivial, inevitable ambiguities based on an algebraic method, as long as there are more sensors than frequencies. We then analyze the sensitivity of this algebraic technique with respect to the number of snapshots and noise. We next propose an optimization approach that makes full use of the measurements by minimizing a non-convex objective which is non-negative and continuously differentiable over all calibration parameters and Toeplitz matrices. We prove that, in the case of infinite snapshots and noiseless measurements, the objective vanishes only at equivalent solutions to the true calibration parameters and the measurement covariance matrix. The objective is minimized using Wirtinger gradient descent which is proven to converge to a critical point. We show empirically that this critical point provides a good approximation of the true calibration parameters and the underlying frequencies.

cs.IT↗

arXiv2018

Undersampled windowed exponentials and their applications

We characterize the completeness and frame/basis property of a union of under-sampled windowed exponentials of the form $$ {\mathcal F}(g): =\{e^{2πi n x}: n\ge 0\}\cup \{g(x)e^{2πi nx}: n<0\} $$ for $L^2[-1/2,1/2]$ by the spectra of the Toeplitz operators with symbol $g$. Using this characterization, we classify all real-valued functions $g$ such that ${\mathcal F}(g)$ is complete or forms a frame/basis. Conversely, we use the classical Kadec-1/4-theorem in non-harmonic Fourier series to determine all $ξ$ such that the Toeplitz operators with symbol $e^{2πi ξx}$ is injective or invertible. These results demonstrate an elegant interaction between frame theory of windowed exponentials and Toeplitz operators. Finally, as an application, we use our results to answer some open questions in dynamical sampling, phase retrieval and derivative samplings on $\ell^2({\mathbb Z})$ and Paley-Wiener spaces of bandlimited functions.

math.FA↗

arXiv2017

Phaseless Reconstruction from Space-Time Samples

Phaseless reconstruction from space-time samples is a nonlinear problem of recovering a function $x$ in a Hilbert space $\mathcal{H}$ from the modulus of linear measurements $\{\lvert \langle x, ϕ_i\rangle \rvert$, $ \ldots$, $\lvert \langle A^{L_i}x, ϕ_i \rangle \rvert : i \in\mathscr I\}$, where $\{ϕ_i; i \in\mathscr I\}\subset \mathcal{H}$ is a set of functionals on $\mathcal{H}$, and $A$ is a bounded operator on $\mathcal{H}$ that acts as an evolution operator. In this paper, we provide various sufficient or necessary conditions for solving this problem, which has connections to $X$-ray crystallography, the scattering transform, and deep learning.

math.FA↗

arXiv2017

Universal Spatiotemporal Sampling Sets for Discrete Spatially Invariant Evolution Systems

Let $(I,+)$ be a finite abelian group and $\mathbf{A}$ be a circular convolution operator on $\ell^2(I)$. The problem under consideration is how to construct minimal $Ω\subset I$ and $l_i$ such that $Y=\{\mathbf{e}_i, \mathbf{A}\mathbf{e}_i, \cdots, \mathbf{A}^{l_i}\mathbf{e}_i: i\in Ω\}$ is a frame for $\ell^2(I)$, where $\{\mathbf{e}_i: i\in I\}$ is the canonical basis of $\ell^2(I)$. This problem is motivated by the spatiotemporal sampling problem in discrete spatially invariant evolution systems. We will show that the cardinality of $Ω$ should be at least equal to the largest geometric multiplicity of eigenvalues of $\mathbf{A}$, and we consider the universal spatiotemporal sampling sets $(Ω, l_i)$ for convolution operators $\mathbf{A}$ with eigenvalues subject to the same largest geometric multiplicity. We will give an algebraic characterization for such sampling sets and show how this problem is linked with sparse signal processing theory and polynomial interpolation theory.

cs.IT↗

arXiv2015

System Identification in Dynamical Sampling

We consider the problem of spatiotemporal sampling in a discrete infinite dimensional spatially invariant evolutionary process $x^{(n)}=A^nx$ to recover an unknown convolution operator $A$ given by a filter $a \in \ell^1(\mathbb{Z})$ and an unknown initial state $x$ modeled as avector in $\ell^2(\mathbb{Z})$. Traditionally, under appropriate hypotheses, any $x$ can be recovered from its samples on $\mathbb{Z}$ and $A$ can be recovered by the classical techniques of deconvolution. In this paper, we will exploit the spatiotemporal correlation and propose a new spatiotemporal sampling scheme to recover $A$ and $x$ that allows to sample the evolving states $x,Ax, \cdots, A^{N-1}x$ on a sub-lattice of $\mathbb{Z}$, and thus achieve the spatiotemporal trade off. The spatiotemporal trade off is motivated by several industrial applications \cite{Lv09}. Specifically, we show that $\{x(m\mathbb{Z}), Ax(m\mathbb{Z}), \cdots, A^{N-1}x(m\mathbb{Z}): N \geq 2m\}$ contains enough information to recover a typical "low pass filter" $a$ and $x$ almost surely, in which we generalize the idea of the finite dimensional case in \cite{AK14}. In particular, we provide an algorithm based on a generalized Prony method for the case when both $a$ and $x$ are of finite impulse response and an upper bound of their support is known. We also perform the perturbation analysis based on the spectral properties of the operator $A$ and initial state $x$, and verify them by several numerical experiments. Finally, we provide several other numerical methods to stabilize the method and numerical example shows the improvement.

cs.IT↗