Searcharxiv⌕ Search

arXiv subjects

Tryphon T. Georgiou

Publications and source records attributed to Tryphon T. Georgiou.

105 records · Page 6Linked to original sources
arXiv2014

Metrics for matrix-valued measures via test functions

It is perhaps not widely recognized that certain common notions of distance between probability measures have an alternative dual interpretation which compares corresponding functionals against suitable families of test functions. This dual viewpoint extends in a straightforward manner to suggest metrics between matrix-valued measures. Our main interest has been in developing weakly-continuous metrics that are suitable for comparing matrix-valued power spectral density functions. To this end, and following the suggested recipe of utilizing suitable families of test functions, we develop a weakly-continuous metric that is analogous to the Wasserstein metric and applies to matrix-valued densities. We use a numerical example to compare this metric to certain standard alternatives including a different version of a matricial Wasserstein metric developed earlier.

eess.SY↗
arXiv2013

Convex Clustering via Optimal Mass Transport

We consider approximating distributions within the framework of optimal mass transport and specialize to the problem of clustering data sets. Distances between distributions are measured in the Wasserstein metric. The main problem we consider is that of approximating sample distributions by ones with sparse support. This provides a new viewpoint to clustering. We propose different relaxations of a cardinality function which penalizes the size of the support set. We establish that a certain relaxation provides the tightest convex lower approximation to the cardinality penalty. We compare the performance of alternative relaxations on a numerical study on clustering.

eess.SY↗
arXiv2013

On time-reversibility of linear stochastic models

Reversal of the time direction in stochastic systems driven by white noise has been central throughout the development of stochastic realization theory, filtering and smoothing. Similar ideas were developed in connection with certain problems in the theory of moments, where a duality induced by time reversal was introduced to parametrize solutions. In this latter work it was shown that stochastic systems driven by arbitrary second-order stationary processes can be similarly time-reversed. By combining these two sets of ideas we present herein a generalization of time-reversal in stochastic realization theory.

eess.SY↗
arXiv2013

Linear models based on noisy data and the Frisch scheme

We address the problem of identifying linear relations among variables based on noisy measurements. This is, of course, a central question in problems involving "Big Data." Often a key assumption is that measurement errors in each variable are independent. This precise formulation has its roots in the work of Charles Spearman in 1904 and of Ragnar Frisch in the 1930's. Various topics such as errors-in-variables, factor analysis, and instrumental variables, all refer to alternative formulations of the problem of how to account for the anticipated way that noise enters in the data. In the present paper we begin by describing the basic theory and provide alternative modern proofs to some key results. We then go on to consider certain generalizations of the theory as well applying certain novel numerical techniques to the problem. A central role is played by the Frisch-Kalman dictum which aims at a noise contribution that allows a maximal set of simultaneous linear relations among the noise-free variables --a rank minimization problem. In the years since Frisch's original formulation, there have been several insights including trace minimization as a convenient heuristic to replace rank minimization. We discuss convex relaxations and certificates guaranteeing global optimality. A complementary point of view to the Frisch-Kalman dictum is introduced in which models lead to a min-max quadratic estimation error for the error-free variables. Points of contact between the two formalisms are discussed and various alternative regularization schemes are indicated.

eess.SY↗
arXiv2013

Matrix-valued Monge-Kantorovich Optimal Mass Transport

We formulate an optimal transport problem for matrix-valued density functions. This is pertinent in the spectral analysis of multivariable time-series. The "mass" represents energy at various frequencies whereas, in addition to a usual transportation cost across frequencies, a cost of rotation is also taken into account. We show that it is natural to seek the transportation plan in the tensor product of the spaces for the two matrix-valued marginals. In contrast to the classical Monge-Kantorovich setting, the transportation plan is no longer supported on a thin zero-measure set.

eess.SY↗
arXiv2012

Uncertainty Bounds for Spectral Estimation

The purpose of this paper is to study metrics suitable for assessing uncertainty of power spectra when these are based on finite second-order statistics. The family of power spectra which is consistent with a given range of values for the estimated statistics represents the uncertainty set about the "true" power spectrum. Our aim is to quantify the size of this uncertainty set using suitable notions of distance, and in particular, to compute the diameter of the set since this represents an upper bound on the distance between any choice of a nominal element in the set and the "true" power spectrum. Since the uncertainty set may contain power spectra with lines and discontinuities, it is natural to quantify distances in the weak topology---the topology defined by continuity of moments. We provide examples of such weakly-continuous metrics and focus on particular metrics for which we can explicitly quantify spectral uncertainty. We then consider certain high resolution techniques which utilize filter-banks for pre-processing, and compute worst-case a priori uncertainty bounds solely on the basis of the filter dynamics. This allows the a priori tuning of the filter-banks for improved resolution over selected frequency bands.

eess.SY↗
arXiv2012

The Separation Principle in Stochastic Control, Redux

Over the last 50 years a steady stream of accounts have been written on the separation principle of stochastic control. Even in the context of the linear-quadratic regulator in continuous time with Gaussian white noise, subtle difficulties arise, unexpected by many, that are often overlooked. In this paper we propose a new framework for establishing the separation principle. This approach takes the viewpoint that stochastic systems are well-defined maps between sample paths rather than stochastic processes per se and allows us to extend the separation principle to systems driven by martingales with possible jumps. While the approach is more in line with "real-life" engineering thinking where signals travel around the feedback loop, it is unconventional from a probabilistic point of view in that control laws for which the feedback equations are satisfied almost surely, and not deterministically for every sample path, are excluded.

math.OC↗
arXiv2011

Distances and Riemannian metrics for multivariate spectral densities

We first introduce a class of divergence measures between power spectral density matrices. These are derived by comparing the suitability of different models in the context of optimal prediction. Distances between "infinitesimally close" power spectra are quadratic, and hence, they induce a differential-geometric structure. We study the corresponding Riemannian metrics and, for a particular case, provide explicit formulae for the corresponding geodesics and geodesic distances. The close connection between the geometry of power spectra and the geometry of the Fisher-Rao metric is noted.

math.OC↗
arXiv2008

Feedback Control and the Arrow of Time

The purpose of this paper is to highlight the central role that the time asymmetry of stability plays in feedback control. We show that this provides a new perspective on the use of doubly-infinite or semi-infinite time axes for signal spaces in control theory. We then focus on the implication of this time asymmetry in modeling uncertainty, regulation and robust control. We point out that modeling uncertainty and the ease of control depend critically on the direction of time. We also discuss the relationship of this control-based time-arrow with the well known arrows of time in physics.

math.DS↗
arXiv2006

An intrinsic metric for power spectral density functions

We present an intrinsic metric that quantifies distances between power spectral density functions. The metric was derived by the author in a recent arXiv-report (math.OC/0607026) as the geodesic distance between spectral density functions with respect to a particular pseudo-Riemannian metric motivated by a quadratic prediction problem. We provide an independent verification of the metric inequality and discuss certain key properties of the induced topology.

math.OC↗
arXiv2006

Distances between power spectral densities

We present several natural notions of distance between spectral density functions of (discrete-time) random processes. They are motivated by certain filtering problems. First we quantify the degradation of performance of a predictor which is designed for a particular spectral density function and then it is used to predict the values of a random process having a different spectral density. The logarithm of the ratio between the variance of the error, over the corresponding minimal (optimal) variance, produces a measure of distance between the two power spectra with several desirable properties. Analogous quantities based on smoothing problems produce alternative distances and suggest a class of measures based on fractions of generalized means of ratios of power spectral densities. These distance measures endow the manifold of spectral density functions with a (pseudo) Riemannian metric. We pursue one of the possible options for a distance measure, characterize the relevant geodesics, and compute corresponding distances.

math.OC↗
arXiv2006

The maximum entropy ansatz in the absence of a time arrow: fractional pole models

The maximum entropy ansatz, as it is often invoked in the context of time-series analysis, suggests the selection of a power spectrum which is consistent with autocorrelation data and corresponds to a random process least predictable from past observations. We introduce and compare a class of spectra with the property that the underlying random process is least predictable at any given point from the complete set of past and future observations. In this context, randomness is quantified by the size of the corresponding smoothing error and deterministic processes are characterized by integrability of the inverse of their power spectral densities--as opposed to the log-integrability in the classical setting. The power spectrum which is consistent with a partial autocorrelation sequence and corresponds to the most random process in this new sense, is no longer rational but generated by finitely many fractional-poles.

math.PR↗
arXiv2005

The Caratheodory-Fejer-Pisarenko decomposition and its multivariable counterpart

When a covariance matrix with a Toeplitz structure is written as the sum of a singular one and a positive scalar multiple of the identity, the singular summand corresponds to the covariance of a purely deterministic component of a time-series whereas the identity corresponds to white noise--this is the Carathéodory-Fejér-Pisarenko (CFP) decomposition. In the present paper we study multivariable analogs for block-Toeplitz matrices as well as for matrices with the structure of state-covariances of finite-dimensional linear systems (which include block-Toeplitz ones). We characterize state-covariances which admit only a deterministic input power spectrum. We show that multivariable decomposition of a state-covariance in accordance with a ``deterministic component + white noise'' hypothesis for the input does not exist in general, and develop formulae for spectra corresponding to singular covariances via decomposing the contribution of the singular part. We consider replacing the ``scalar multiple of the identity'' in the CFP decomposition by a covariance of maximal trace which is admissible as a summand. The summand can be either (block-)diagonal corresponding to white noise or have a ``short-range correlation structure'' correponding to a moving average component. The trace represents the maximal variance/energy that can be accounted for by a process (e.g., noise) with the aforementioned structure at the input, and the optimal solution can be computed via convex optimization. The decomposition of covariances and spectra according to the range of their time-domain correlations is an alternative to the CFP-dictum with potentially great practical significance.

math.OC↗
arXiv2005

Relative entropy and the multi-variable multi-dimensional moment problem

Entropy-like functionals on operator algebras have been studied since the pioneering work of von Neumann, Umegaki, Lindblad, and Lieb. The most well-known are the von Neumann entropy $trace (ρ\log ρ)$ and a generalization of the Kullback-Leibler distance $trace (ρ\log ρ- ρ\log σ)$, refered to as quantum relative entropy and used to quantify distance between states of a quantum system. The purpose of this paper is to explore these as regularizing functionals in seeking solutions to multi-variable and multi-dimensional moment problems. It will be shown that extrema can be effectively constructed via a suitable homotopy. The homotopy approach leads naturally to a further generalization and a description of all the solutions to such moment problems. This is accomplished by a renormalization of a Riemannian metric induced by entropy functionals. As an application we discuss the inverse problem of describing power spectra which are consistent with second-order statistics, which has been the main motivation behind the present work.

math.OC↗