arXiv subjects

Waheed U. Bajwa

Publications and source records attributed to Waheed U. Bajwa.

At least 55 records · Page 3Linked to original sources

arXiv2016

Conditioning of Random Block Subdictionaries with Applications to Block-Sparse Recovery and Regression

The linear model, in which a set of observations is assumed to be given by a linear combination of columns of a matrix, has long been the mainstay of the statistics and signal processing literature. One particular challenge for inference under linear models is understanding the conditions on the dictionary under which reliable inference is possible. This challenge has attracted renewed attention in recent years since many modern inference problems deal with the "underdetermined" setting, in which the number of observations is much smaller than the number of columns in the dictionary. This paper makes several contributions for this setting when the set of observations is given by a linear combination of a small number of groups of columns of the dictionary, termed the "block-sparse" case. First, it specifies conditions on the dictionary under which most block subdictionaries are well conditioned. This result is fundamentally different from prior work on block-sparse inference because (i) it provides conditions that can be explicitly computed in polynomial time, (ii) the given conditions translate into near-optimal scaling of the number of columns of the block subdictionaries as a function of the number of observations for a large class of dictionaries, and (iii) it suggests that the spectral norm and the quadratic-mean block coherence of the dictionary (rather than the worst-case coherences) fundamentally limit the scaling of dimensions of the well-conditioned block subdictionaries. Second, this paper investigates the problems of block-sparse recovery and block-sparse regression in underdetermined settings. Near-optimal block-sparse recovery and regression are possible for certain dictionaries as long as the dictionary satisfies easily computable conditions and the coefficients describing the linear combination of groups of columns can be modeled through a mild statistical prior.

math.ST↗

arXiv2016

Minimax Lower Bounds for Kronecker-Structured Dictionary Learning

Dictionary learning is the problem of estimating the collection of atomic elements that provide a sparse representation of measured/collected signals or data. This paper finds fundamental limits on the sample complexity of estimating dictionaries for tensor data by proving a lower bound on the minimax risk. This lower bound depends on the dimensions of the tensor and parameters of the generative model. The focus of this paper is on second-order tensor data, with the underlying dictionaries constructed by taking the Kronecker product of two smaller dictionaries and the observed data generated by sparse linear combinations of dictionary atoms observed through white Gaussian noise. In this regard, the paper provides a general lower bound on the minimax risk and also adapts the proof techniques for equivalent results using sparse and Gaussian coefficient models. The reported results suggest that the sample complexity of dictionary learning for tensor data can be significantly lower than that for unstructured data.

cs.IT↗

arXiv2016

Design and Analysis Framework for Sparse FIR Channel Shortening

A major performance and complexity limitation in broadband communications is the long channel delay spread which results in a highly-frequency-selective channel frequency response. Channel shortening equalizers (CSEs) are used to ensure that the cascade of a long channel impulse response (CIR) and the CSE is approximately equivalent to a target impulse response (TIR) with much shorter delay spread. In this paper, we propose a general framework that transforms the problems of design of sparse CSE and TIR finite impulse response (FIR) filters into the problem of sparsest-approximation of a vector in different dictionaries. In addition, we compare several choices of sparsifying dictionaries under this framework. Furthermore, the worst-case coherence of these dictionaries, which determines their sparsifying effectiveness, are analytically and/or numerically evaluated. Finally, the usefulness of the proposed framework for the design of sparse CSE and TIR filters is validated through numerical experiments.

cs.IT↗

arXiv2015

A General Framework for the Design and Analysis of Sparse FIR Linear Equalizers

Complexity of linear finite-impulse-response (FIR) equalizers is proportional to the square of the number of nonzero taps in the filter. This makes equalization of channels with long impulse responses using either zero-forcing or minimum mean square error (MMSE) filters computationally expensive. Sparse equalization is a widely-used technique to solve this problem. In this paper, a general framework is provided that transforms the problem of sparse linear equalizers (LEs) design into the problem of sparsest-approximation of a vector in different dictionaries. In addition, some possible choices of sparsifying dictionaries in this framework are discussed. Furthermore, the worst-case coherence of some of these dictionaries, which determines their sparsifying strength, are analytically and/or numerically evaluated. Finally, the usefulness of the proposed framework for the design of sparse FIR LEs is validated through numerical experiments.

cs.IT↗

arXiv2015

Cloud K-SVD: A Collaborative Dictionary Learning Algorithm for Big, Distributed Data

This paper studies the problem of data-adaptive representations for big, distributed data. It is assumed that a number of geographically-distributed, interconnected sites have massive local data and they are interested in collaboratively learning a low-dimensional geometric structure underlying these data. In contrast to previous works on subspace-based data representations, this paper focuses on the geometric structure of a union of subspaces (UoS). In this regard, it proposes a distributed algorithm---termed cloud K-SVD---for collaborative learning of a UoS structure underlying distributed data of interest. The goal of cloud K-SVD is to learn a common overcomplete dictionary at each individual site such that every sample in the distributed data can be represented through a small number of atoms of the learned dictionary. Cloud K-SVD accomplishes this goal without requiring exchange of individual samples between sites. This makes it suitable for applications where sharing of raw data is discouraged due to either privacy concerns or large volumes of data. This paper also provides an analysis of cloud K-SVD that gives insights into its properties as well as deviations of the dictionaries learned at individual sites from a centralized solution in terms of different measures of local/global data and topology of interconnections. Finally, the paper numerically illustrates the efficacy of cloud K-SVD on real and synthetic distributed data.

cs.LG↗

arXiv2015

Learning the nonlinear geometry of high-dimensional data: Models and algorithms

Modern information processing relies on the axiom that high-dimensional data lie near low-dimensional geometric structures. This paper revisits the problem of data-driven learning of these geometric structures and puts forth two new nonlinear geometric models for data describing "related" objects/phenomena. The first one of these models straddles the two extremes of the subspace model and the union-of-subspaces model, and is termed the metric-constrained union-of-subspaces (MC-UoS) model. The second one of these models---suited for data drawn from a mixture of nonlinear manifolds---generalizes the kernel subspace model, and is termed the metric-constrained kernel union-of-subspaces (MC-KUoS) model. The main contributions of this paper in this regard include the following. First, it motivates and formalizes the problems of MC-UoS and MC-KUoS learning. Second, it presents algorithms that efficiently learn an MC-UoS or an MC-KUoS underlying data of interest. Third, it extends these algorithms to the case when parts of the data are missing. Last, but not least, it reports the outcomes of a series of numerical experiments involving both synthetic and real data that demonstrate the superiority of the proposed geometric models and learning algorithms over existing approaches in the literature. These experiments also help clarify the connections between this work and the literature on (subspace and kernel k-means) clustering.

stat.ML↗

arXiv2015

Identification of Linear Time-Varying Systems Through Waveform Diversity

Linear, time-varying (LTV) systems composed of time shifts, frequency shifts, and complex amplitude scalings are operators that act on continuous finite-energy waveforms. This paper presents a novel, resource-efficient method for identifying the parametric description of such systems, i.e., the time shifts, frequency shifts, and scalings, from the sampled response to linear frequency modulated (LFM) waveforms, with emphasis on the application to radar processing. If the LTV operator is probed with a sufficiently diverse set of LFM waveforms, then the system can be identified with high accuracy. In the case of noiseless measurements, the identification is perfect, while in the case of noisy measurements, the accuracy is inversely proportional to the noise level. The use of parametric estimation techniques with recently proposed denoising algorithms allows the estimation of the parameters with high accuracy.

cs.IT↗

arXiv2014

MIMO-MC Radar: A MIMO Radar Approach Based on Matrix Completion

In a typical MIMO radar scenario, transmit nodes transmit orthogonal waveforms, while each receive node performs matched filtering with the known set of transmit waveforms, and forwards the results to the fusion center. Based on the data it receives from multiple antennas, the fusion center formulates a matrix, which, in conjunction with standard array processing schemes, such as MUSIC, leads to target detection and parameter estimation. In MIMO radars with compressive sensing (MIMO-CS), the data matrix is formulated by each receive node forwarding a small number of compressively obtained samples. In this paper, it is shown that under certain conditions, in both sampling cases, the data matrix at the fusion center is low-rank, and thus can be recovered based on knowledge of a small subset of its entries via matrix completion (MC) techniques. Leveraging the low-rank property of that matrix, we propose a new MIMO radar approach, termed, MIMO-MC radar, in which each receive node either performs matched filtering with a small number of randomly selected dictionary waveforms or obtains sub-Nyquist samples of the received signal at random sampling instants, and forwards the results to a fusion center. Based on the received samples, and with knowledge of the sampling scheme, the fusion center partially fills the data matrix and subsequently applies MC techniques to estimate the full matrix. MIMO-MC radars share the advantages of the recently proposed MIMO-CS radars, i.e., high resolution with reduced amounts of data, but unlike MIMO-CS radars do not require grid discretization. The MIMO-MC radar concept is illustrated through a linear uniform array configuration, and its target estimation performance is demonstrated via simulations.

cs.IT↗

arXiv2013

Bits Through Bufferless Queues

This paper investigates the capacity of a channel in which information is conveyed by the timing of consecutive packets passing through a queue with independent and identically distributed service times. Such timing channels are commonly studied under the assumption of a work-conserving queue. In contrast, this paper studies the case of a bufferless queue that drops arriving packets while a packet is in service. Under this bufferless model, the paper provides upper bounds on the capacity of timing channels and establishes achievable rates for the case of bufferless M/M/1 and M/G/1 queues. In particular, it is shown that a bufferless M/M/1 queue at worst suffers less than 10% reduction in capacity when compared to an M/M/1 work-conserving queue.

cs.IT↗

arXiv2013

Posterior consistency in linear models under shrinkage priors

We investigate the asymptotic behavior of posterior distributions of regression coefficients in high-dimensional linear models as the number of dimensions grows with the number of observations. We show that the posterior distribution concentrates in neighborhoods of the true parameter under simple sufficient conditions. These conditions hold under popular shrinkage priors given some sparsity assumptions.

stat.ME↗

arXiv2013

Level set estimation from projection measurements: Performance guarantees and fast computation

Estimation of the level set of a function (i.e., regions where the function exceeds some value) is an important problem with applications in digital elevation mapping, medical imaging, astronomy, etc. In many applications, the function of interest is not observed directly. Rather, it is acquired through (linear) projection measurements, such as tomographic projections, interferometric measurements, coded-aperture measurements, and random projections associated with compressed sensing. This paper describes a new methodology for rapid and accurate estimation of the level set from such projection measurements. The key defining characteristic of the proposed method, called the projective level set estimator, is its ability to estimate the level set from projection measurements without an intermediate reconstruction step. This leads to significantly faster computation relative to heuristic "plug-in" methods that first estimate the function, typically with an iterative algorithm, and then threshold the result. The paper also includes a rigorous theoretical analysis of the proposed method, which utilizes the recent results from the non-asymptotic theory of random matrices results from the literature on concentration of measure and characterizes the estimator's performance in terms of geometry of the measurement operator and 1-norm of the discretized function.

stat.AP↗

arXiv2013

Target Estimation in Colocated MIMO Radar via Matrix Completion

We consider a colocated MIMO radar scenario, in which the receive antennas forward their measurements to a fusion center. Based on the received data, the fusion center formulates a matrix which is then used for target parameter estimation. When the receive antennas sample the target returns at Nyquist rate, and assuming that there are more receive antennas than targets, the data matrix at the fusion center is low-rank. When each receive antenna sends to the fusion center only a small number of samples, along with the sample index, the receive data matrix has missing elements, corresponding to the samples that were not forwarded. Under certain conditions, matrix completion techniques can be applied to recover the full receive data matrix, which can then be used in conjunction with array processing techniques, e.g., MUSIC, to obtain target information. Numerical results indicate that good target recovery can be achieved with occupancy of the receive data matrix as low as 50%.

cs.IT↗

arXiv2013

Finding Zeros: Greedy Detection of Holes

In this paper, motivated by the setting of white-space detection [1], we present theoretical and empirical results for detection of the zero-support E of x \in Cp (xi = 0 for i \in E) with reduced-dimension linear measurements. We propose two low- complexity algorithms based on one-step thresholding [2] for this purpose. The second algorithm is a variant of the first that further assumes the presence of group-structure in the target signal [3] x. Performance guarantees for both algorithms based on the worst- case and average coherence (group coherence) of the measurement matrix is presented along with the empirical performance of the algorithms.

cs.IT↗

arXiv2013

Toward Resource-Optimal Consensus over the Wireless Medium

We carry out a comprehensive study of the resource cost of averaging consensus in wireless networks. Most previous approaches suppose a graphical network, which abstracts away crucial features of the wireless medium, and measure resource consumption only in terms of the total number of transmissions required to achieve consensus. Under a path-loss dominated model, we study the resource requirements of consensus with respect to three wireless-appropriate metrics: total transmit energy, elapsed time, and time-bandwidth product. First we characterize the performance of several popular gossip algorithms, showing that they may be order-optimal with respect to transmit energy but are strictly suboptimal with respect to elapsed time and time-bandwidth product. Further, we propose a new consensus scheme, termed hierarchical averaging, and show that it is nearly order-optimal with respect to all three metrics. Finally, we examine the effects of quantization, showing that hierarchical averaging provides a nearly order-optimal tradeoff between resource consumption and quantization error.

cs.IT↗

arXiv2012

A Constrained Random Demodulator for Sub-Nyquist Sampling

This paper presents a significant modification to the Random Demodulator (RD) of Tropp et al. for sub-Nyquist sampling of frequency-sparse signals. The modification, termed constrained random demodulator, involves replacing the random waveform, essential to the operation of the RD, with a constrained random waveform that has limits on its switching rate because fast switching waveforms may be hard to generate cleanly. The result is a relaxation on the hardware requirements with a slight, but manageable, decrease in the recovery guarantees. The paper also establishes the importance of properly choosing the statistics of the constrained random waveform. If the power spectrum of the random waveform matches the distribution on the tones of the input signal (i.e., the distribution is proportional to the power spectrum), then recovery of the input signal tones is improved. The theoretical guarantees provided in the paper are validated through extensive numerical simulations and phase transition plots.

cs.IT↗

arXiv2012

Group Model Selection Using Marginal Correlations: The Good, the Bad and the Ugly

Group model selection is the problem of determining a small subset of groups of predictors (e.g., the expression data of genes) that are responsible for majority of the variation in a response variable (e.g., the malignancy of a tumor). This paper focuses on group model selection in high-dimensional linear models, in which the number of predictors far exceeds the number of samples of the response variable. Existing works on high-dimensional group model selection either require the number of samples of the response variable to be significantly larger than the total number of predictors contributing to the response or impose restrictive statistical priors on the predictors and/or nonzero regression coefficients. This paper provides comprehensive understanding of a low-complexity approach to group model selection that avoids some of these limitations. The proposed approach, termed Group Thresholding (GroTh), is based on thresholding of marginal correlations of groups of predictors with the response variable and is reminiscent of existing thresholding-based approaches in the literature. The most important contribution of the paper in this regard is relating the performance of GroTh to a polynomial-time verifiable property of the predictors for the general case of arbitrary (random or deterministic) predictors and arbitrary nonzero regression coefficients.

math.ST↗

arXiv2011

Two are better than one: Fundamental parameters of frame coherence

This paper investigates two parameters that measure the coherence of a frame: worst-case and average coherence. We first use worst-case and average coherence to derive near-optimal probabilistic guarantees on both sparse signal detection and reconstruction in the presence of noise. Next, we provide a catalog of nearly tight frames with small worst-case and average coherence. Later, we find a new lower bound on worst-case coherence; we compare it to the Welch bound and use it to interpret recently reported signal reconstruction results. Finally, we give an algorithm that transforms frames in a way that decreases average coherence without changing the spectral norm or worst-case coherence.

math.FA↗

arXiv2011

Asynchronous Code-Division Random Access Using Convex Optimization

Many applications in cellular systems and sensor networks involve a random subset of a large number of users asynchronously reporting activity to a base station. This paper examines the problem of multiuser detection (MUD) in random access channels for such applications. Traditional orthogonal signaling ignores the random nature of user activity in this problem and limits the total number of users to be on the order of the number of signal space dimensions. Contention-based schemes, on the other hand, suffer from delays caused by colliding transmissions and the hidden node problem. In contrast, this paper presents a novel pairing of an asynchronous non-orthogonal code-division random access scheme with a convex optimization-based MUD algorithm that overcomes the issues associated with orthogonal signaling and contention-based methods. Two key distinguishing features of the proposed MUD algorithm are that it does not require knowledge of the delay or channel state information of every user and it has polynomial-time computational complexity. The main analytical contribution of this paper is the relationship between the performance of the proposed MUD algorithm in the presence of arbitrary or random delays and two simple metrics of the set of user codewords. The study of these metrics is then focused on two specific sets of codewords, random binary codewords and specially constructed algebraic codewords, for asynchronous random access. The ensuing analysis confirms that the proposed scheme together with either of these two codeword sets significantly outperforms the orthogonal signaling-based random access in terms of the total number of users in the system.

cs.IT↗