SearcharxivSearch

arXiv subjects

Giorgio Picci

Publications and source records attributed to Giorgio Picci.

17 recordsLinked to original sources

On Irreversibility and Stochastic Systems: Part One

We attempt to characterize irreversibility of a dynamical system from the existence of different forward and backward mathematical representations depending on the direction of the time arrow. Such different representations have been studied intensively and are shown to exist for stochastic diffusion models. In this setting one has however to face the preliminary justification of stochastic description for physical systems which are described by classical mechanics as inherently deterministic and conservative. In part one of this paper we first address this modeling problem for linear systems in a deterministic context. We show that forward-backward representations can also describe conservative finite dimensional deterministic systems when they are coupled to an infinite-dimensional conservative heat bath. A novel key observation is that the heat bath acts on the finite-dimensional conservative system by {\em state-feedback} and can shift its eigenvalues to make the system dissipative but may also generate another totally unstable model which naturally evolves backward in time. In the second part, we address the stochastic description of these two representations. Under a natural family of invariant measures the heat bath can be shown to induce a white noise input acting on the system making it look like a true dissipative diffusion.

math.DS

An Empirical Bayes approach to ARX Estimation

Empirical Bayes inference is based on estimation of the parameters of an a priori distribution from the observed data. The estimation technique of the parameters of the prior, called hyperparameters, is based on the marginal distribution obtained by integrating the joint density of the model with respect to the prior. This is a key step which needs to be properly adapted to the problem at hand. In this paper we study Empirical Bayes inference of linear autoregressive models with inputs (ARX models) for time series and compare the performance of the marginal parametric estimator with that a full Empirical Bayesian analysis based on the estimated prior. Such a comparison, can only make sense for a (realistic) finite data length. In this setting, we propose a new estimation technique of the hyperparameters by a sequential Bayes procedure which is essentially a backward Kalman filter. It turns out that for finite data length the marginal Bayes tends to behave slightly better than the full Empirical Bayesian parameter estimator and so also in the case of slowly varying random parameters.

eess.SY

On Irreversibility and Stochastic Systems; Part Two

We attempt to characterize irreversibility of a dynamical system from the existence of different forward and backward mathematical representations depending on the direction of the time arrow. Such different representations have been studied intensively and are shown to exist for stochastic diffusion models. In this setting one has however to face the preliminary justification of the existence of a stochastic description for physical systems which are traditionally described by classical mechanics as inherently deterministic and conservative. In part one of this paper we have addressed this modeling problem from a deterministic viewpoint for linear systems. We have shown that there are forward-backward representations which can describe conservative finite dimensional deterministic systems when they are coupled to an infinite-dimensional conservative heat bath. A key observation is that the heat bath acts on the finite-dimensional system by {\em state-feedback} which can shift its eigenvalues to make the system dissipative, but also may generate a totally unstable system which naturally evolves backward in time. In this second part, we address the stochastic description of these two representations. Under a natural family of invariant measures it is shown that the heat bath induces a white noise input acting on both the forward-backward representations making them true dissipative diffusions. We also consider how to relate the Stochastics to the Hamiltonian deterministic picture discussed in Part one. In our current context, we show that a continuous stationary process with a rational spectrum can always be thought of, or represented as, the output of a lossless deterministic system coupled to an infinite dimensional heat bath.

math-ph

Modeling of Low Rank Time Series

Rank-deficient stationary stochastic vector processes are present in many problems in network theory and dynamic factor analysis. In this paper we study hidden dynamical relations between the components of a discrete-time stochastic vector process and investigate their properties with respect to stability and causality. More specifically, we construct transfer functions with a full-rank input process formed from selected components of the given vector process and having a vector process of the remaining components as output. An important question, which we answer in the negative, is whether it is always possible to find such a deterministic relation that is stable. If it is unstable, there must be feedback from output to input ensuring that stationarity is maintained. This leads to connections to robust control. We also show how our results could be used to investigate the structure of dynamic network models and the latent low-rank stochastic process in a dynamic factor model.

eess.SY

Identification of Low Rank Vector Processes

We study modeling and identification of stationary processes with a spectral density matrix of low rank. Equivalently, we consider processes having an innovation of reduced dimension for which Prediction Error Methods (PEM) algorithms are not directly applicable. We show that these processes admit a special feedback structure with a deterministic feedback channel which can be used to split the identification in two steps, one of which can be based on standard algorithms while the other is based on a deterministic least squares fit. Identifiability of the feedback system is analyzed and a unique identifiable structure is characterized. Simulations show that the proposed procedure works well in some simple examples.

eess.SY

Hidden Factor estimation in Dynamic Generalized Factor Analysis Models

This paper deals with the estimation of the hidden factor in Dynamic Generalized Factor Analysis via a generalization of Kalman filtering. Asymptotic consistency is discussed and it is shown that the Kalman one-step predictor is not the right tool while the pure filter yields a consistent estimate.

math.ST

Modeling and Identification of Low Rank Vector Processes

We study modeling and identification of processes with a spectral density matrix of low rank. Equivalently, we consider processes having an innovation of reduced dimension for which Prediction Error Methods (PEM) algorithms are not directly applicable. We show that these processes admit a special feedback structure with a deterministic feedback channel which can be used to split the identification in two steps, one of which can be based on standard algorithms while the other is based on a deterministic least squares fit.

eess.SY

An Empirical Bayes Approach to Frequency Estimation

In this paper we show that the classical problem of frequency estimation can be formulated and solved efficiently in an empirical Bayesian framework by assigning a uniform a priori probability distribution to the unknown frequency. We discover that the a posteriori covariance matrix of the signal model is the discrete-time counterpart of an operator whose eigenfunctions are the famous prolate spheroidal wave functions, introduced by Slepian and coworkers in the 1960's and widely studied in the signal processing literature although motivated by a different class of problems. The special structure of the covariance matrix is exploited to design an estimator for the hyperparameters of the prior distribution which is essentially linear, based on subspace identification. Bayesian analysis based on the estimated prior then shows that the estimated center-frequency is asymptotically coincident with the MAP estimate. This stochastic approach leads to consistent estimates, provides uncertainty bounds and may advantageously supersede standard parametric estimation methods which are based on iterative optimization algorithms of local nature. Simulations show that the approach is quite promising and seems to compare favorably with some classical methods.

eess.SP

Approximation of stationary processes and Toeplitz Spectra

We study the approximation of stationary processes by a simple class of purely deterministic signals. This has an analytic counterpart in the approximation of symmetric positive definite Toeplitz matrices by submatrices of finite rank. We propose a notion of distance between them and prove a weak sense approximation result.

math.PR

Spectral Rank, Feedback, Causality and the Indirect Method for CARMA Identification

Building on a recent paper by Georgiou and Lindquist [1] on the problem of rank deficiency of spectral densities and hidden dynamical relations after sampling of continuous-time stochastic processes, this paper is devoted to understanding related questions of feedback and Granger causality that affect stability properties. This then naturally connects to CARMA identification, where we remark on certain oversights in the literature.

math.DS

On the state space and dynamics selection in linear stochastic models: a spectral factorization approach

Matrix spectral factorization is traditionally described as finding spectral factors having a fixed analytic pole configuration. The classification of spectral factors then involves studying the solutions of a certain algebraic Riccati equation which parametrizes their zero structure. The pole structure of the spectral factors can be also parametrized in terms of solutions of another Riccati equation. We study the relation between the solution sets of these two Riccati equations and describe the construction of general spectral factors which involve both zero- and pole-flipping on an arbitrary reference spectral factor.

eess.SY

A New Algorithm for Circulant Rational Covariance Extension and Applications to Finite-interval Smoothing

The partial stochastic realization of periodic processes from finite covariance data has recently been solved by Lindquist and Picci based on convex optimization of a generalized entropy functional. The meaning and the role of this criterion have an unclear origin. In this paper we propose a solution based on a nonlinear generalization of the classical Yule-Walker type equations and on a new iterative algorithm which is shown to converge to the same (unique) solution of the variational problem. This provides a conceptual link to the variational principles and at the same time yields a robust algorithm which can for example be successfully applied to finite-interval smoothing problems providing a simpler procedure if compared with the classical Riccati-based calculations.

stat.ME

Modeling of Stationary Periodic Time Series by ARMA Representations

This is a survey of some recent results on the rational circulant covariance extension problem: Given a partial sequence $(c_0,c_1,\dots,c_n)$ of covariance lags $c_k=\mathbb{E}\{y(t+k)\overline{y(t)}\}$ emanating from a stationary periodic process $\{y(t)\}$ with period $2N>2n$, find all possible rational spectral functions of $\{y(t)\}$ of degree at most $2n$ or, equivalently, all bilateral and unilateral ARMA models of order at most $n$, having this partial covariance sequence. Each representation is obtained as the solution of a pair of dual convex optimization problems. This theory is then reformulated in terms of circulant matrices and the connections to reciprocal processes and the covariance selection problem is explained. Next it is shown how the theory can be extended to the multivariate case. Finally, an application to image processing is presented.

math.ST

Representation and Factorization of Discrete-Time Rational All-Pass Functions

We obtain a general characterization of discrete-time all-pass rational matrix functions from state-space representations. It can be employed to address model reduction problems in the same vein of the theory developed by Glover in the continuous-time. Besides model reduction, this characterization is shown to be useful in a variety of contexts such as studying LMI's and Riccati equations and especially in the factorization of all-pass functions. The results are obtained in the most general setting, without introducing any {\em ad hoc} assumption.

math.OC

Modeling complex systems by Generalized Factor Analysis

We propose a new modeling paradigm for large dimensional aggregates of stochastic systems by Generalized Factor Analysis (GFA) models. These models describe the data as the sum of a flocking plus an uncorrelated idiosyncratic component. The flocking component describes a sort of collective orderly motion which admits a much simpler mathematical description than the whole ensemble while the idiosyncratic component describes weakly correlated noise. We first discuss static GFA representations and characterize in a rigorous way the properties of the two components. The extraction of the dynamic flocking component is discussed for time-stationary linear systems and for a simple classes of separable random fields.

eess.SY

The Circulant Rational Covariance Extension Problem: The Complete Solution

The rational covariance extension problem to determine a rational spectral density given a finite number of covariance lags can be seen as a matrix completion problem to construct an infinite-dimensional positive-definite Toeplitz matrix the north-west corner of which is given. The circulant rational covariance extension problem considered in this paper is a modification of this problem to partial stochastic realization of reciprocal and periodic stationary process, which are better represented on the discrete unit circle $\mathbb{Z}_{2N}$ rather than on the discrete real line $\mathbb{Z}$. The corresponding matrix completion problem then amounts to completing a finite-dimensional Toeplitz matrix that is circulant. Another important motivation for this problem is that it provides a natural approximation, involving only computations based on the fast Fourier transform, for the ordinary rational covariance extension problem, potentially leading to an efficient numerical procedure for the latter. The circulant rational covariance extension problem is an inverse problem with infinitely many solutions in general, each corresponding to a bilateral ARMA representation of the underlying periodic (reciprocal) process. In this paper we present a complete smooth parameterization of all solutions and convex optimization procedures for determining them. A procedure to determine which solution that best matches additional data in the form of logarithmic moments is also presented.

math.OC

A Maximum Entropy solution of the Covariance Extension Problem for Reciprocal Processes

Stationary reciprocal processes defined on a finite interval of the integer line can be seen as a special class of Markov random fields restricted to one dimension. Non stationary reciprocal processes have been extensively studied in the past especially by Jamison, Krener, Levy and co-workers. The specialization of the non-stationary theory to the stationary case, however, does not seem to have been pursued in sufficient depth in the literature. Stationary reciprocal processes (and reciprocal stochastic models) are potentially useful for describing signals which naturally live in a finite region of the time (or space) line. Estimation or identification of these models starting from observed data seems still to be an open problem which can lead to many interesting applications in signal and image processing. In this paper, we discuss a class of reciprocal processes which is the acausal analog of auto-regressive (AR) processes, familiar in control and signal processing. We show that maximum likelihood identification of these processes leads to a covariance extension problem for block-circulant covariance matrices. This generalizes the famous covariance band extension problem for stationary processes on the integer line. As in the usual stationary setting on the integer line, the covariance extension problem turns out to be a basic conceptual and practical step in solving the identification problem. We show that the maximum entropy principle leads to a complete solution of the problem.

math.OC