arXiv subjects

Ljubisa Stankovic

Publications and source records attributed to Ljubisa Stankovic.

36 records · Page 2Linked to original sources

arXiv2020

RANSAC-Based Signal Denoising Using Compressive Sensing

In this paper, we present an approach to the reconstruction of signals exhibiting sparsity in a transformation domain, having some heavily disturbed samples. This sparsity-driven signal recovery exploits a carefully suited random sampling consensus (RANSAC) methodology for the selection of an inlier subset of samples. To this aim, two fundamental properties are used: a signal sample represents a linear combination of the sparse coefficients, whereas the disturbance degrade original signal sparsity. The properly selected samples are further used as measurements in the sparse signal reconstruction, performed using algorithms from the compressive sensing framework. Besides the fact that the disturbance degrades signal sparsity in the transformation domain, no other disturbance-related assumptions are made -- there are no special requirements regarding its statistical behavior or the range of its values. As a case study, the discrete Fourier transform (DFT) is considered as a domain of signal sparsity, owing to its significance in signal processing theory and applications. Numerical results strongly support the presented theory. In addition, exact relation for the signal-to-noise ratio (SNR) of the reconstructed signal is also presented. This simple result, which conveniently characterizes the RANSAC-based reconstruction performance, is numerically confirmed by a set of statistical examples.

cs.IT↗

arXiv2020

A Class of Doubly Stochastic Shift Operators for Random Graph Signals and their Boundedness

A class of doubly stochastic graph shift operators (GSO) is proposed, which is shown to exhibit: (i) lower and upper $L_{2}$-boundedness for locally stationary random graph signals; (ii) $L_{2}$-isometry for \textit{i.i.d.} random graph signals with the asymptotic increase in the incoming neighbourhood size of vertices; and (iii) preservation of the mean of any graph signal. These properties are obtained through a statistical consistency analysis of the graph shift, and by exploiting the dual role of the doubly stochastic GSO as a Markov (diffusion) matrix and as an unbiased expectation operator. Practical utility of the class of doubly stochastic GSOs is demonstrated in a real-world multi-sensor signal filtering setting.

eess.SP↗

arXiv2020

Graph Signal Processing -- Part III: Machine Learning on Graphs, from Graph Topology to Applications

Many modern data analytics applications on graphs operate on domains where graph topology is not known a priori, and hence its determination becomes part of the problem definition, rather than serving as prior knowledge which aids the problem solution. Part III of this monograph starts by addressing ways to learn graph topology, from the case where the physics of the problem already suggest a possible topology, through to most general cases where the graph topology is learned from the data. A particular emphasis is on graph topology definition based on the correlation and precision matrices of the observed data, combined with additional prior knowledge and structural conditions, such as the smoothness or sparsity of graph connections. For learning sparse graphs (with small number of edges), the least absolute shrinkage and selection operator, known as LASSO is employed, along with its graph specific variant, graphical LASSO. For completeness, both variants of LASSO are derived in an intuitive way, and explained. An in-depth elaboration of the graph topology learning paradigm is provided through several examples on physically well defined graphs, such as electric circuits, linear heat transfer, social and computer networks, and spring-mass systems. As many graph neural networks (GNN) and convolutional graph networks (GCN) are emerging, we have also reviewed the main trends in GNNs and GCNs, from the perspective of graph signal filtering. Tensor representation of lattice-structured graphs is next considered, and it is shown that tensors (multidimensional data arrays) are a special class of graph signals, whereby the graph vertices reside on a high-dimensional regular lattice structure. This part of monograph concludes with two emerging applications in financial data processing and underground transportation networks modeling.

cs.IT↗

arXiv2019

Vertex-Frequency Graph Signal Processing: A review

Graph signal processing deals with signals which are observed on an irregular graph domain. While many approaches have been developed in classical graph theory to cluster vertices and segment large graphs in a signal independent way, signal localization based approaches to the analysis of data on graph represent a new research direction which is also a key to big data analytics on graphs. To this end, after an overview of the basic definitions in graphs and graph signals, we present and discuss a localized form of the graph Fourier transform. To establish an analogy with classical signal processing, spectral- and vertex-domain definitions of the localization window are given next. The spectral and vertex localization kernels are then related to the wavelet transform, followed by a study of filtering and inversion of the localized graph Fourier transform. For rigour, the analysis of energy representation and frames in the localized graph Fourier transform is extended to the energy forms of vertex-frequency distributions, which operate even without the need to apply localization windows. Another link with classical signal processing is established through the concept of local smoothness, which is subsequently related to the particular paradigm of signal smoothness on graphs. This all represents a comprehensive account of the relation of general vertex-frequency analysis with classical time-frequency analysis, and important but missing link for more advanced applications of graphs signal processing. The theory is supported by illustrative and practically relevant examples.

eess.SP↗

arXiv2019

Portfolio Cuts: A Graph-Theoretic Framework to Diversification

Investment returns naturally reside on irregular domains, however, standard multivariate portfolio optimization methods are agnostic to data structure. To this end, we investigate ways for domain knowledge to be conveniently incorporated into the analysis, by means of graphs. Next, to relax the assumption of the completeness of graph topology and to equip the graph model with practically relevant physical intuition, we introduce the portfolio cut paradigm. Such a graph-theoretic portfolio partitioning technique is shown to allow the investor to devise robust and tractable asset allocation schemes, by virtue of a rigorous graph framework for considering smaller, computationally feasible, and economically meaningful clusters of assets, based on graph cuts. In turn, this makes it possible to fully utilize the asset returns covariance matrix for constructing the portfolio, even without the requirement for its inversion. The advantages of the proposed framework over traditional methods are demonstrated through numerical simulations based on real-world price data.

eess.SP↗

arXiv2019

Graph Signal Processing -- Part II: Processing and Analyzing Signals on Graphs

The focus of Part I of this monograph has been on both the fundamental properties, graph topologies, and spectral representations of graphs. Part II embarks on these concepts to address the algorithmic and practical issues centered round data/signal processing on graphs, that is, the focus is on the analysis and estimation of both deterministic and random data on graphs. The fundamental ideas related to graph signals are introduced through a simple and intuitive, yet illustrative and general enough case study of multisensor temperature field estimation. The concept of systems on graph is defined using graph signal shift operators, which generalize the corresponding principles from traditional learning systems. At the core of the spectral domain representation of graph signals and systems is the Graph Discrete Fourier Transform (GDFT). The spectral domain representations are then used as the basis to introduce graph signal filtering concepts and address their design, including Chebyshev polynomial approximation series. Ideas related to the sampling of graph signals are presented and further linked with compressive sensing. Localized graph signal analysis in the joint vertex-spectral domain is referred to as the vertex-frequency analysis, since it can be considered as an extension of classical time-frequency analysis to the graph domain of a signal. Important topics related to the local graph Fourier transform (LGFT) are covered, together with its various forms including the graph spectral and vertex domain windows and the inversion conditions and relations. A link between the LGFT with spectral varying window and the spectral graph wavelet transform (SGWT) is also established. Realizations of the LGFT and SGWT using polynomial (Chebyshev) approximations of the spectral functions are further considered. Finally, energy versions of the vertex-frequency representations are introduced.

cs.IT↗

arXiv2019

Unitary Shift Operators on a Graph

A unitary shift operator (GSO) for signals on a graph is introduced, which exhibits the desired property of energy preservation over both backward and forward graph shifts. For rigour, the graph differential operator is also derived in an analytical form. The commutativity relation of the shift operator with the Fourier transform is next explored in conjunction with the proposed GSO to introduce a graph discrete Fourier transform (GDFT) which, unlike existing approaches, ensures the orthogonality of GDFT bases and admits a natural frequency-domain interpretation. The proposed GDFT is shown to allow for a coherent definition of the graph discrete Hilbert transform (GDHT) and the graph analytic signal. The advantages of the proposed GSO are demonstrated through illustrative examples.

eess.SP↗

arXiv2019

Graph Signal Processing -- Part I: Graphs, Graph Spectra, and Spectral Clustering

The area of Data Analytics on graphs promises a paradigm shift as we approach information processing of classes of data, which are typically acquired on irregular but structured domains (social networks, various ad-hoc sensor networks). Yet, despite its long history, current approaches mostly focus on the optimization of graphs themselves, rather than on directly inferring learning strategies, such as detection, estimation, statistical and probabilistic inference, clustering and separation from signals and data acquired on graphs. To fill this void, we first revisit graph topologies from a Data Analytics point of view, and establish a taxonomy of graph networks through a linear algebraic formalism of graph topology (vertices, connections, directivity). This serves as a basis for spectral analysis of graphs, whereby the eigenvalues and eigenvectors of graph Laplacian and adjacency matrices are shown to convey physical meaning related to both graph topology and higher-order graph properties, such as cuts, walks, paths, and neighborhoods. Next, to illustrate estimation strategies performed on graph signals, spectral analysis of graphs is introduced through eigenanalysis of mathematical descriptors of graphs and in a generic way. Finally, a framework for vertex clustering and graph segmentation is established based on graph spectral representation (eigenanalysis) which illustrates the power of graphs in various data association tasks. The supporting examples demonstrate the promise of Graph Data Analytics in modeling structural and functional/semantic inferences. At the same time, Part I serves as a basis for Part II and Part III which deal with theory, methods and applications of processing Data on Graphs and Graph Topology Learning from data.

cs.IT↗

arXiv2019

Quantization in Compressive Sensing: A Signal Processing Approach

Influence of the finite-length registers and quantization effects on the reconstruction of sparse and approximately sparse signals is analyzed in this paper. For the nonquantized measurements, the compressive sensing (CS) framework provides highly accurate reconstruction algorithms that produce negligible errors when the reconstruction conditions are met. However, hardware implementations of signal processing algorithms involve the finite-length registers and quantization of the measurements. An analysis of the effects related to the measurements quantization with an arbitrary number of bits is the topic of this paper. A unified mathematical model for the analysis of the quantization noise and the signal nonsparsity on the CS reconstruction is presented. An exact formula for the expected energy of error in the CS-based reconstructed signal is derived. The theory is validated through various numerical examples with quantized measurements, including the cases of approximately sparse signals, noise folding, and floating-point arithmetics.

cs.IT↗

arXiv2019

An Example-Driven Introduction to Data Analytics on Graphs

Graphs are irregular structures which naturally account for data integrity, however, traditional approaches have been established outside Signal Processing, and largely focus on analyzing the underlying graphs rather than signals on graphs. Given the rapidly increasing availability of multisensor and multinode measurements, likely recorded on irregular or ad-hoc grids, it would be extremely advantageous to analyze such structured data as graph signals and thus benefit from the ability of graphs to incorporate spatial awareness of the sensing locations, sensor importance, and local versus global sensor association. The aim of this lecture note is therefore to establish a common language between graph signals, defined on irregular signal domains, and some of the most fundamental paradigms in DSP, such as spectral analysis of multichannel signals, system transfer function, digital filter design, parameter estimation, and optimal filter design. This is achieved through a physically meaningful and intuitive real-world example of geographically distributed multisensor temperature estimation. A similar spatial multisensor arrangement is already widely used in Signal Processing curricula to introduce minimum variance estimators and Kalman filters \cite{HM}, and by adopting this framework we facilitate a seamless integration of graph theory into the curriculum of existing DSP courses. By bridging the gap between standard approaches and graph signal processing, we also show that standard methods can be thought of as special cases of their graph counterparts, evaluated on linear graphs. It is hoped that our approach would not only help to demystify graph theoretic approaches in education and research but it would also empower practitioners to explore a whole host of otherwise prohibitive modern applications.

eess.SP↗

arXiv2019

On the Decomposition of Multivariate Nonstationary Multicomponent Signals

With their ability to handle an increased amount of information, multivariate and multichannel signals can be used to solve problems normally not solvable with signals obtained from a single source. One such problem is the decomposition signals with several components whose domains of support significantly overlap in both the time and the frequency domain, including the joint time-frequency domain. Initially, we proposed a solution to this problem based on the Wigner distribution of multivariate signals, which requires the attenuation of the cross-terms. In this paper, an advanced solution based on an eigenvalue analysis of the multivariate signal autocorrelation matrix, followed by their time-frequency concentration measure minimization, is presented. This analysis provides less restrictive conditions for the signal decomposition than in the case of Wigner distribution. The algorithm for the components separation is based on the concentration measures of the eigenvector time-frequency representation, that are linear combinations of the overlapping signal components. With an increased number of sensors/channels, the robustness of the decomposition process to additive noise is also achieved. The theory is supported by numerical examples. The required channel dissimilarity is statistically investigated as well.

cs.IT↗

arXiv2019

An Intuitive Derivation of the Coherence Index Relation in Compressive Sensing

The existence and uniqueness conditions are a prerequisite for reliable reconstruction of sparse signals from reduced sets of measurements within the Compressive Sensing (CS) paradigm. However, despite their underpinning role for practical applications, existing uniqueness relations are either computationally prohibitive to implement (Restricted Isometry Property), or involve mathematical tools that are beyond the standard background of engineering graduates (Coherence Index). This can introduce conceptual and computational obstacles in the development of engineering intuition, the design of suboptimal practical solutions, or understanding of limitations. To this end, we introduce a simple but rigorous derivation of the coherence index condition, based on standard linear algebra, with the aim to empower signal processing practitioners with intuition in the design and ease in implementation of CS systems. Given that the coherence index is one of very few CS metrics that admits mathematically tractable and computationally feasible calculation, it is our hope that this work will help bridge the gap between the theory and applications of compressive sensing.

cs.IT↗

arXiv2017

Conditions for Unique Reconstruction of Sparse Signals Using Compressive Sensing Methods

A signal is sparse in one of its representation domain if the number of nonzero coefficients in that domain is much smaller than the total number of coefficients. Sparse signals can be reconstructed from a very reduced set of measurements/observations. The topic of this paper are conditions for the unique reconstruction of sparse signals from a reduced set of observations. After the basic definitions are introduced, the unique reconstruction conditions are reviewed using the spark, restricted isometry, and coherence of the measurement matrix. Uniqueness of the reconstruction of signals sparse in the discrete Fourier domain (DFT), as the most important signal transformation domain, is considered as well.

cs.IT↗

arXiv2015

Reconstruction of Sparse and Nonsparse Signals from a Reduced Set of Samples

Signals sparse in a transformation domain can be recovered from a reduced set of randomly positioned samples by using compressive sensing algorithms. Simple re- construction algorithms are presented in the first part of the paper. The missing samples manifest themselves as a noise in this reconstruction. Once the reconstruction conditions for a sparse signal are met and the reconstruction is achieved, the noise due to missing samples does not influence the results in a direct way. It influences the possibility to recover a signal only. Additive input noise will remain in the resulting reconstructed signal. The accuracy of the recovery results is related to the additive input noise. Simple derivation of this relation is presented. If a reconstruction algorithm for a sparse signal is used in the reconstruction of a nonsparse signal then the noise due to missing samples will remain and behave as an additive input noise. An exact relation for the mean square error of this error is derived for the partial DFT matrix case in this paper and presented in form of a theorem. It takes into account very important fact that if all samples are available then the error will be zero, for both sparse and nonsparse recovered signals. Theory is illustrated and checked on statistical examples.

cs.IT↗

arXiv2015

Reconstruction of Randomly Sampled Sparse Signals Using an Adaptive Gradient Algorithm

Sparse signals can be recovered from a reduced set of samples by using compressive sensing algorithms. In common methods the signal is recovered in the sparse domain. A method for the reconstruction of sparse signal which reconstructs the remaining missing samples/measurements is recently proposed. The available samples are fixed, while the missing samples are considered as minimization variables. Recovery of missing samples/measurements is done using an adaptive gradient-based algorithm in the time domain. A new criterion for the parameter adaptation in this algorithm, based on the gradient direction angles, is proposed. It improves the algorithm computational efficiency. A theorem for the uniqueness of the recovered signal for given set of missing samples (reconstruction variables) is presented. The case when available samples are a random subset of a uniformly or nonuniformly sampled signal is considered in this paper. A recalculation procedure is used to reconstruct the nonuniformly sampled signal. The methods are illustrated on statistical examples.

cs.IT↗

arXiv2015

Polynomial Fourier Domain as a Domain of Signal Sparsity

A compressive sensing (CS) reconstruction method for polynomial phase signals is proposed in this paper. It relies on the Polynomial Fourier transform, which is used to establish a relationship between the observation and sparsity domain. Polynomial phase signals are not sparse in commonly used domains such as Fourier or wavelet domain. Therefore, for polynomial phase signals standard CS algorithms applied in these transformation domains cannot provide satisfactory results. In that sense, the Polynomial Fourier transform is used to ensure sparsity. The proposed approach is generalized using time-frequency representations obtained by the Local Polynomial Fourier transform (LPFT). In particular, the first-order LPFT can produce linear time-frequency representation for chirps. It provides revealing signal local behavior, which leads to sparse representation. The theory is illustrated on examples.

cs.IT↗

arXiv2014

On the ISAR Image Analysis and Recovery with Unavailable or Heavily Corrupted Data

Common ISAR radar images and signals can be reconstructed from much fewer samples than the sampling theorem requires since they are usually sparse. Unavailable randomly positioned samples can result from heavily corrupted parts of the signal. Since these samples can be omitted and declared as unavailable, the application of the compressive sensing methods in the recovery of heavily corrupted signal and radar images is possible. A\ simple direct method for the recovery of unavailable signal samples and the calculation of the restored ISAR image is reviewed. An analysis of the noise influence is performed. For fast maneuvering ISAR targets the sparsity property is lost since the ISAR image is blurred. A nonparametric quadratic time-frequency representations based method is used to restore the ISAR image sparsity. However, the linear relation between the signal and the sparsity domain transformation is lost. A recently proposed gradient recovery algorithm is adapted for this kind of analysis. It does not require the linear relation of the signal and its sparsity domain transformation in the process of unavailable data recovery. The presented methods and results are tested on several numerical examples proving the expected accuracy and improvements.

cs.IT↗

arXiv2013

Adaptive Variable Step Algorithm for Missing Samples Recovery in Sparse Signals

Recovery of arbitrarily positioned samples that are missing in sparse signals recently attracted significant research interest. Sparse signals with heavily corrupted arbitrary positioned samples could be analyzed in the same way as compressive sensed signals by omitting the corrupted samples and considering them as unavailable during the recovery process. The reconstruction of missing samples is done by using one of the well known reconstruction algorithms. In this paper we will propose a very simple and efficient adaptive variable step algorithm, applied directly to the concentration measures, without reformulating the reconstruction problem within the standard linear programming form. Direct application of the gradient approach to the nondifferentiable forms of measures lead us to introduce a variable step size algorithm. A criterion for changing adaptive algorithm parameters is presented. The results are illustrated on the examples with sparse signals, including approximately sparse signals and noisy sparse signals.

cs.IT↗