arXiv subjects

Cédric Richard

Publications and source records attributed to Cédric Richard.

48 records · Page 3Linked to original sources

arXiv2015

Diffusion LMS over Multitask Networks

The diffusion LMS algorithm has been extensively studied in recent years. This efficient strategy allows to address distributed optimization problems over networks in the case where nodes have to collaboratively estimate a single parameter vector. Problems of this type are referred to as single-task problems. Nevertheless, there are several problems in practice that are multitask-oriented in the sense that the optimum parameter vector may not be the same for every node. This brings up the issue of studying the performance of the diffusion LMS algorithm when it is run, either intentionally or unintentionally, in a multitask environment. In this paper, we conduct a theoretical analysis on the stochastic behavior of diffusion LMS in the case where the so-called single-task hypothesis is violated. We explain under what conditions diffusion LMS continues to deliver performance superior to non-cooperative strategies in the multitask environment. When the conditions are violated, we explain how to endow the nodes with the ability to cluster with other similar nodes to remove bias. We propose an unsupervised clustering strategy that allows each node to select, via adaptive adjustments of combination weights, the neighboring nodes with which it can collaborate to estimate a common parameter vector. Simulations are presented to illustrate the theoretical results, and to demonstrate the efficiency of the proposed clustering strategy. The framework is applied to a useful problem involving a multi-target tracking task.

eess.SY↗

arXiv2014

A graph Laplacian regularization for hyperspectral data unmixing

This paper introduces a graph Laplacian regularization in the hyperspectral unmixing formulation. The proposed regularization relies upon the construction of a graph representation of the hyperspectral image. Each node in the graph represents a pixel's spectrum, and edges connect spectrally and spatially similar pixels. The proposed graph framework promotes smoothness in the estimated abundance maps and collaborative estimation between homogeneous areas of the image. The resulting convex optimization problem is solved using the Alternating Direction Method of Multipliers (ADMM). A special attention is given to the computational complexity of the algorithm, and Graph-cut methods are proposed in order to reduce the computational burden. Finally, simulations conducted on synthetic data illustrate the effectiveness of the graph Laplacian regularization with respect to other classical regularizations for hyperspectral unmixing.

cs.CV↗

arXiv2014

Blind and fully constrained unmixing of hyperspectral images

This paper addresses the problem of blind and fully constrained unmixing of hyperspectral images. Unmixing is performed without the use of any dictionary, and assumes that the number of constituent materials in the scene and their spectral signatures are unknown. The estimated abundances satisfy the desired sum-to-one and nonnegativity constraints. Two models with increasing complexity are developed to achieve this challenging task, depending on how noise interacts with hyperspectral data. The first one leads to a convex optimization problem, and is solved with the Alternating Direction Method of Multipliers. The second one accounts for signal-dependent noise, and is addressed with a Reweighted Least Squares algorithm. Experiments on synthetic and real data demonstrate the effectiveness of our approach.

stat.AP↗

arXiv2014

Steady-state performance of non-negative least-mean-square algorithm and its variants

Non-negative least-mean-square (NNLMS) algorithm and its variants have been proposed for online estimation under non-negativity constraints. The transient behavior of the NNLMS, Normalized NNLMS, Exponential NNLMS and Sign-Sign NNLMS algorithms have been studied in our previous work. In this technical report, we derive closed-form expressions for the steady-state excess mean-square error (EMSE) for the four algorithms. Simulations results illustrate the accuracy of the theoretical results. This is a complementary material to our previous work.

cs.LG↗

arXiv2013

Multitask Diffusion Adaptation over Networks

Adaptive networks are suitable for decentralized inference tasks, e.g., to monitor complex natural phenomena. Recent research works have intensively studied distributed optimization problems in the case where the nodes have to estimate a single optimum parameter vector collaboratively. However, there are many important applications that are multitask-oriented in the sense that there are multiple optimum parameter vectors to be inferred simultaneously, in a collaborative manner, over the area covered by the network. In this paper, we employ diffusion strategies to develop distributed algorithms that address multitask problems by minimizing an appropriate mean-square error criterion with $\ell_2$-regularization. The stability and convergence of the algorithm in the mean and in the mean-square sense is analyzed. Simulations are conducted to verify the theoretical findings, and to illustrate how the distributed strategy can be used in several useful applications related to spectral sensing, target localization, and hyperspectral data unmixing.

cs.MA↗

arXiv2013

Nonlinear unmixing of hyperspectral images using a semiparametric model and spatial regularization

Incorporating spatial information into hyperspectral unmixing procedures has been shown to have positive effects, due to the inherent spatial-spectral duality in hyperspectral scenes. Current research works that consider spatial information are mainly focused on the linear mixing model. In this paper, we investigate a variational approach to incorporating spatial correlation into a nonlinear unmixing procedure. A nonlinear algorithm operating in reproducing kernel Hilbert spaces, associated with an $\ell_1$ local variation norm as the spatial regularizer, is derived. Experimental results, with both synthetic and real data, illustrate the effectiveness of the proposed scheme.

stat.ML↗

arXiv2013

Diffusion LMS for clustered multitask networks

Recent research works on distributed adaptive networks have intensively studied the case where the nodes estimate a common parameter vector collaboratively. However, there are many applications that are multitask-oriented in the sense that there are multiple parameter vectors that need to be inferred simultaneously. In this paper, we employ diffusion strategies to develop distributed algorithms that address clustered multitask problems by minimizing an appropriate mean-square error criterion with $\ell_2$-regularization. Some results on the mean-square stability and convergence of the algorithm are also provided. Simulations are conducted to illustrate the theoretical findings.

eess.SY↗

arXiv2013

Convergence analysis of kernel LMS algorithm with pre-tuned dictionary

The kernel least-mean-square (KLMS) algorithm is an appealing tool for online identification of nonlinear systems due to its simplicity and robustness. In addition to choosing a reproducing kernel and setting filter parameters, designing a KLMS adaptive filter requires to select a so-called dictionary in order to get a finite-order model. This dictionary has a significant impact on performance, and requires careful consideration. Theoretical analysis of KLMS as a function of dictionary setting has rarely, if ever, been addressed in the literature. In an analysis previously published by the authors, the dictionary elements were assumed to be governed by the same probability density function of the input data. In this paper, we modify this study by considering the dictionary as part of the filter parameters to be set. This theoretical analysis paves the way for future investigations on KLMS dictionary design.

stat.ML↗

arXiv2013

A normalized scaled gradient method to solve non-negativity and equality constrained linear inverse problem - Application to spectral mixture analysis

This paper addresses the problem of minimizing a convex cost function under non-negativity and equality constraints, with the aim of solving the linear unmixing problem encountered in hyperspectral imagery. This problem can be formulated as a linear regression problem whose regression coefficients (abundances) satisfy sum-to-one and positivity constraints. A normalized scaled gradient iterative method (NSGM) is proposed for estimating the abundances of the linear mixing model. The positivity constraint is ensured by the Karush Kuhn Tucker conditions whereas the sum-to-one constraint is fulfilled by introducing normalized variables in the algorithm. The convergence is ensured by a one-dimensional search of the step size. Note that NSGM can be applied to any convex cost function with non negativity and flux constraints. In order to compare the NSGM with the well-known fully constraint least squares (FCLS) algorithm, this latter is reformulated in term of a penalized function, which reveals its suboptimality. Simulations on synthetic data illustrate the performances of the proposed algorithm in comparison with other unmixing algorithms and, more particulary, demonstrate its efficiency when compared to the popular FCLS. Finally, results on real data are given.

math.OC↗

arXiv2013

Nonlinear unmixing of hyperspectral images: models and algorithms

When considering the problem of unmixing hyperspectral images, most of the literature in the geoscience and image processing areas relies on the widely used linear mixing model (LMM). However, the LMM may be not valid and other nonlinear models need to be considered, for instance, when there are multi-scattering effects or intimate interactions. Consequently, over the last few years, several significant contributions have been proposed to overcome the limitations inherent in the LMM. In this paper, we present an overview of recent advances in nonlinear unmixing modeling.

physics.data-an↗

arXiv2013

Online dictionary learning for kernel LMS. Analysis and forward-backward splitting algorithm

Adaptive filtering algorithms operating in reproducing kernel Hilbert spaces have demonstrated superiority over their linear counterpart for nonlinear system identification. Unfortunately, an undesirable characteristic of these methods is that the order of the filters grows linearly with the number of input data. This dramatically increases the computational burden and memory requirement. A variety of strategies based on dictionary learning have been proposed to overcome this severe drawback. Few, if any, of these works analyze the problem of updating the dictionary in a time-varying environment. In this paper, we present an analytical study of the convergence behavior of the Gaussian least-mean-square algorithm in the case where the statistics of the dictionary elements only partially match the statistics of the input data. This allows us to emphasize the need for updating the dictionary in an online way, by discarding the obsolete elements and adding appropriate ones. We introduce a kernel least-mean-square algorithm with L1-norm regularization to automatically perform this task. The stability in the mean of this method is analyzed, and its performance is tested with experiments.

stat.ML↗

arXiv2013

Distributed dictionary learning over a sensor network

We consider the problem of distributed dictionary learning, where a set of nodes is required to collectively learn a common dictionary from noisy measurements. This approach may be useful in several contexts including sensor networks. Diffusion cooperation schemes have been proposed to solve the distributed linear regression problem. In this work we focus on a diffusion-based adaptive dictionary learning strategy: each node records observations and cooperates with its neighbors by sharing its local dictionary. The resulting algorithm corresponds to a distributed block coordinate descent (alternate optimization). Beyond dictionary learning, this strategy could be adapted to many matrix factorization problems and generalized to various settings. This article presents our approach and illustrates its efficiency on some numerical examples.

stat.ML↗