Lorenzo Finesso | Searcharxiv

arXiv2024

Synchronous deautoconvolution algorithm for discrete-time positive signals via I-divergence approximation

We pose the problem of the optimal approximation of a given nonnegative signal $y_t$ with the scalar autoconvolution $(x*x)_t$ of a nonnegative signal $x_t$, where $x_t$ and $y_t$ are signals of equal length. The $\mathcal{I}$-divergence has been adopted as optimality criterion, being well suited to incorporate nonnegativity constraints. To find a minimizer we derive an iterative descent algorithm of the alternating minimization type. The algorithm is based on the lifting of the original problem to a larger space, a relaxation technique developed by Csiszár and Tusnády in [Statistics \& Decisions (S1) (1984), 205--237] which, in the present context, requires the solution of a hard partial minimization problem. We study the asymptotic behavior of the algorithm exploiting the optimality properties of the partial minimization problems and prove, among other results, that its limit points are Kuhn-Tucker points of the original minimization problem. Numerical experiments illustrate the results.

math.OC↗

arXiv2021

The inverse problem of positive autoconvolution

We pose the problem of approximating optimally a given nonnegative signal with the scalar autoconvolution of a nonnegative signal. The I-divergence is chosen as the optimality criterion being well suited to incorporate nonnegativity constraints. After proving the existence of an optimal approximation we derive an iterative descent algorithm of the alternating minimization type to find a minimizer. The algorithm is based on the lifting technique developed by Csiszár and Tusnádi and exploits the optimality properties of the related minimization problems in the larger space. We study the asymptotic behavior of the iterative algorithm and prove, among other results, that its limit points are Kuhn-Tucker points of the original minimization problem. Numerical experiments confirm the asymptotic results and exhibit the fast convergence of the proposed algorithm.

math.OC↗

arXiv2018

Approximation of nonnegative systems by moving averages of fixed order

We pose the approximation problem for scalar nonnegative input-output systems via impulse response convolutions of finite order, i.e. finite order moving averages, based on repeated observations of input/output signal pairs. The problem is converted into a nonnegative matrix factorization with special structure for which we use Csiszár's I-divergence as the criterion of optimality. Conditions are given, on the input/output data, that guarantee the existence and uniqueness of the minimum. We propose an algorithm of the alternating minimization type for I-divergence minimization, and present its asymptotic behavior. For the case of noisy observations we give the large sample properties of the statistical version of the minimization problem for different observation regimes. Numerical experiments confirm the asymptotic results and exhibit fast convergence of the proposed algorithm.

math.OC↗

arXiv2015

Countable Partially Exchangeable Mixtures

Partially exchangeable sequences representable as mixtures of Markov chains are completely specified by de Finetti's mixing measure. The paper characterizes, in terms of a subclass of hidden Markov models, the partially exchangeable sequences with mixing measure concentrated on a countable set, both for discrete valued and for Polish valued sequences.

math.PR↗

arXiv2013

Approximation of Nonnegative Systems by Finite Impulse Response Convolutions

We pose the deterministic, nonparametric, approximation problem for scalar nonnegative input/output systems via finite impulse response convolutions, based on repeated observations of input/output signal pairs. The problem is converted into a nonnegative matrix factorization with special structure for which we use Csiszár's I-divergence as the criterion of optimality. Conditions are given, on the input/output data, that guarantee the existence and uniqueness of the minimum. We propose a standard algorithm of the alternating minimization type for I-divergence minimization, and study its asymptotic behavior. We also provide a statistical version of the minimization problem and give its large sample properties.

math.OC↗

arXiv2010

Approximate factor analysis model building via alternating I-divergence minimization

Given a positive definite covariance matrix $\widehat Σ$, we strive to construct an optimal \emph{approximate} factor analysis model $HH^\top +D$, with $H$ having a prescribed number of columns and $D>0$ diagonal. The optimality criterion we minimize is the I-divergence between the corresponding normal laws. Lifting the problem into a properly chosen larger space enables us to derive an alternating minimization algorithm à la Csiszár-Tusnády for the construction of the best approximation. The convergence properties of the algorithm are studied, with special attention given to the case where $D$ is singular.

math.PR↗

arXiv2008

Approximation of stationary processes by Hidden Markov Models

We aim at the construction of a Hidden Markov Model (HMM) of assigned complexity (number of states of the underlying Markov chain) which best approximates, in Kullback-Leibler divergence rate, a given stationary process. We establish, under mild conditions, the existence of the divergence rate between a stationary process and an HMM. Since in general there is no analytic expression available for this divergence rate, we approximate it with a properly defined, and easily computable, divergence between Hankel matrices, which we use as our approximation criterion. We propose a three-step algorithm, based on the Nonnegative Matrix Factorization technique, which realizes an HMM optimal with respect to the defined approximation criterion. A full theoretical analysis of the algorithm is given in the special case of Markov approximation.

math.OC↗

arXiv2007

Factor Analysis and Alternating Minimization

In this paper we make a first attempt at understanding how to build an optimal approximate normal factor analysis model. The criterion we have chosen to evaluate the distance between different models is the I-divergence between the corresponding normal laws. The algorithm that we propose for the construction of the best approximation is of an the alternating minimization kind.

math.PR↗

arXiv2005

Nonnegative Matrix Factorization and I-Divergence Alternating Minimization

In this paper we consider the Nonnegative Matrix Factorization (NMF) problem: given an (elementwise) nonnegative matrix $V \in \R_+^{m\times n}$ find, for assigned $k$, nonnegative matrices $W\in\R_+^{m\times k}$ and $H\in\R_+^{k\times n}$ such that $V=WH$. Exact, non trivial, nonnegative factorizations do not always exist, hence it is interesting to pose the approximate NMF problem. The criterion which is commonly employed is I-divergence between nonnegative matrices. The problem becomes that of finding, for assigned $k$, the factorization $WH$ closest to $V$ in I-divergence. An iterative algorithm, EM like, for the construction of the best pair $(W, H)$ has been proposed in the literature. In this paper we interpret the algorithm as an alternating minimization procedure à la Csiszár-Tusnády and investigate some of its stability properties. NMF is widespreading as a data analysis method in applications for which the positivity constraint is relevant. There are other data analysis methods which impose some form of nonnegativity: we discuss here the connections between NMF and Archetypal Analysis.

math.OC↗

arXiv2004

Approximate Nonnegative Matrix Factorization via Alternating Minimization

In this paper we consider the Nonnegative Matrix Factorization (NMF) problem: given an (elementwise) nonnegative matrix $V \in \R_+^{m\times n}$ find, for assigned $k$, nonnegative matrices $W\in\R_+^{m\times k}$ and $H\in\R_+^{k\times n}$ such that $V=WH$. Exact, non trivial, nonnegative factorizations do not always exist, hence it is interesting to pose the approximate NMF problem. The criterion which is commonly employed is I-divergence between nonnegative matrices. The problem becomes that of finding, for assigned $k$, the factorization $WH$ closest to $V$ in I-divergence. An iterative algorithm, EM like, for the construction of the best pair $(W, H)$ has been proposed in the literature. In this paper we interpret the algorithm as an alternating minimization procedure à la Csiszár-Tusnády and investigate some of its stability properties. NMF is widespreading as a data analysis method in applications for which the positivity constraint is relevant. There are other data analysis methods which impose some form of nonnegativity: we discuss here the connections between NMF and Archetypal Analysis. An interesting system theoretic application of NMF is to the problem of approximate realization of Hidden Markov Models.

math.OC↗