arXiv subjects

Bhaskar D. Rao

Publications and source records attributed to Bhaskar D. Rao.

At least 37 records · Page 2Linked to original sources

arXiv2018

Rectified Gaussian Scale Mixtures and the Sparse Non-Negative Least Squares Problem

In this paper, we develop a Bayesian evidence maximization framework to solve the sparse non-negative least squares (S-NNLS) problem. We introduce a family of probability densities referred to as the Rectified Gaussian Scale Mixture (R- GSM) to model the sparsity enforcing prior distribution for the solution. The R-GSM prior encompasses a variety of heavy-tailed densities such as the rectified Laplacian and rectified Student- t distributions with a proper choice of the mixing density. We utilize the hierarchical representation induced by the R-GSM prior and develop an evidence maximization framework based on the Expectation-Maximization (EM) algorithm. Using the EM based method, we estimate the hyper-parameters and obtain a point estimate for the solution. We refer to the proposed method as rectified sparse Bayesian learning (R-SBL). We provide four R- SBL variants that offer a range of options for computational complexity and the quality of the E-step computation. These methods include the Markov chain Monte Carlo EM, linear minimum mean-square-error estimation, approximate message passing and a diagonal approximation. Using numerical experiments, we show that the proposed R-SBL method outperforms existing S-NNLS solvers in terms of both signal and support recovery performance, and is also very robust against the structure of the design matrix.

cs.LG↗

arXiv2018

Superfast Line Spectral Estimation

A number of recent works have proposed to solve the line spectral estimation problem by applying off-the-grid extensions of sparse estimation techniques. These methods are preferable over classical line spectral estimation algorithms because they inherently estimate the model order. However, they all have computation times which grow at least cubically in the problem size, thus limiting their practical applicability in cases with large dimensions. To alleviate this issue, we propose a low-complexity method for line spectral estimation, which also draws on ideas from sparse estimation. Our method is based on a Bayesian view of the problem. The signal covariance matrix is shown to have Toeplitz structure, allowing superfast Toeplitz inversion to be used. We demonstrate that our method achieves estimation accuracy at least as good as current methods and that it does so while being orders of magnitudes faster.

eess.SP↗

arXiv2018

Re-Weighted Learning for Sparsifying Deep Neural Networks

This paper addresses the topic of sparsifying deep neural networks (DNN's). While DNN's are powerful models that achieve state-of-the-art performance on a large number of tasks, the large number of model parameters poses serious storage and computational challenges. To combat these difficulties, a growing line of work focuses on pruning network weights without sacrificing performance. We propose a general affine scaling transformation (AST) algorithm to sparsify DNN's. Our approach follows in the footsteps of popular sparse recovery techniques, which have yet to be explored in the context of DNN's. We describe a principled framework for transforming densely connected DNN's into sparsely connected ones without sacrificing network performance. Unlike existing methods, our approach is able to learn sparse connections at each layer simultaneously, and achieves comparable pruning results on the architecture tested.

cs.LG↗

arXiv2018

A Unified Framework for Sparse Non-Negative Least Squares using Multiplicative Updates and the Non-Negative Matrix Factorization Problem

We study the sparse non-negative least squares (S-NNLS) problem. S-NNLS occurs naturally in a wide variety of applications where an unknown, non-negative quantity must be recovered from linear measurements. We present a unified framework for S-NNLS based on a rectified power exponential scale mixture prior on the sparse codes. We show that the proposed framework encompasses a large class of S-NNLS algorithms and provide a computationally efficient inference procedure based on multiplicative update rules. Such update rules are convenient for solving large sets of S-NNLS problems simultaneously, which is required in contexts like sparse non-negative matrix factorization (S-NMF). We provide theoretical justification for the proposed approach by showing that the local minima of the objective function being optimized are sparse and the S-NNLS algorithms presented are guaranteed to converge to a set of stationary points of the objective function. We then extend our framework to S-NMF, showing that our framework leads to many well known S-NMF algorithms under specific choices of prior and providing a guarantee that a popular subclass of the proposed algorithms converges to a set of stationary points of the objective function. Finally, we study the performance of the proposed approaches on synthetic and real-world data.

stat.ML↗

arXiv2017

A GAMP Based Low Complexity Sparse Bayesian Learning Algorithm

In this paper, we present an algorithm for the sparse signal recovery problem that incorporates damped Gaussian generalized approximate message passing (GGAMP) into Expectation-Maximization (EM)-based sparse Bayesian learning (SBL). In particular, GGAMP is used to implement the E-step in SBL in place of matrix inversion, leveraging the fact that GGAMP is guaranteed to converge with appropriate damping. The resulting GGAMP-SBL algorithm is much more robust to arbitrary measurement matrix $\boldsymbol{A}$ than the standard damped GAMP algorithm while being much lower complexity than the standard SBL algorithm. We then extend the approach from the single measurement vector (SMV) case to the temporally correlated multiple measurement vector (MMV) case, leading to the GGAMP-TSBL algorithm. We verify the robustness and computational advantages of the proposed algorithms through numerical experiments.

cs.LG↗

arXiv2017

Relevance Subject Machine: A Novel Person Re-identification Framework

We propose a novel method called the Relevance Subject Machine (RSM) to solve the person re-identification (re-id) problem. RSM falls under the category of Bayesian sparse recovery algorithms and uses the sparse representation of the input video under a pre-defined dictionary to identify the subject in the video. Our approach focuses on the multi-shot re-id problem, which is the prevalent problem in many video analytics applications. RSM captures the essence of the multi-shot re-id problem by constraining the support of the sparse codes for each input video frame to be the same. Our proposed approach is also robust enough to deal with time varying outliers and occlusions by introducing a sparse, non-stationary noise term in the model error. We provide a novel Variational Bayesian based inference procedure along with an intuitive interpretation of the proposed update rules. We evaluate our approach over several commonly used re-id datasets and show superior performance over current state-of-the-art algorithms. Specifically, for ILIDS-VID, a recent large scale re-id dataset, RSM shows significant improvement over all published approaches, achieving an 11.5% (absolute) improvement in rank 1 accuracy over the closest competing algorithm considered.

cs.CV↗

arXiv2017

Performance of Cell-Free Massive MIMO Systems with MMSE and LSFD Receivers

Cell-Free Massive MIMO comprises a large number of distributed single-antenna access points (APs) serving a much smaller number of users. There is no partitioning into cells and each user is served by all APs. In this paper, the uplink performance of cell-free systems with minimum mean squared error (MMSE) and large scale fading decoding (LSFD) receivers is investigated. The main idea of LSFD receiver is to maximize achievable throughput using only large scale fading coefficients between APs and users. Capacity lower bounds for MMSE and LSFD receivers are derived. An asymptotic approximation for signal-to-interference-plus-noise ratio (SINR) of MMSE receiver is derived as a function of large scale fading coefficients only. The obtained approximation is accurate even for a small number of antennas. MMSE and LSFD receivers demonstrate five-fold and two-fold gains respectively over matched filter (MF) receiver in terms of 5%-outage rate.

cs.IT↗

arXiv2016

Robust Bayesian Method for Simultaneous Block Sparse Signal Recovery with Applications to Face Recognition

In this paper, we present a novel Bayesian approach to recover simultaneously block sparse signals in the presence of outliers. The key advantage of our proposed method is the ability to handle non-stationary outliers, i.e. outliers which have time varying support. We validate our approach with empirical results showing the superiority of the proposed method over competing approaches in synthetic data experiments as well as the multiple measurement face recognition problem.

cs.CV↗

arXiv2015

Type I and Type II Bayesian Methods for Sparse Signal Recovery using Scale Mixtures

In this paper, we propose a generalized scale mixture family of distributions, namely the Power Exponential Scale Mixture (PESM) family, to model the sparsity inducing priors currently in use for sparse signal recovery (SSR). We show that the successful and popular methods such as LASSO, Reweighted $\ell_1$ and Reweighted $\ell_2$ methods can be formulated in an unified manner in a maximum a posteriori (MAP) or Type I Bayesian framework using an appropriate member of the PESM family as the sparsity inducing prior. In addition, exploiting the natural hierarchical framework induced by the PESM family, we utilize these priors in a Type II framework and develop the corresponding EM based estimation algorithms. Some insight into the differences between Type I and Type II methods is provided and of particular interest in the algorithmic development is the Type II variant of the popular and successful reweighted $\ell_1$ method. Extensive empirical results are provided and they show that the Type II methods exhibit better support recovery than the corresponding Type I methods.

cs.LG↗

arXiv2014

Spatiotemporal Sparse Bayesian Learning with Applications to Compressed Sensing of Multichannel Physiological Signals

Energy consumption is an important issue in continuous wireless telemonitoring of physiological signals. Compressed sensing (CS) is a promising framework to address it, due to its energy-efficient data compression procedure. However, most CS algorithms have difficulty in data recovery due to non-sparsity characteristic of many physiological signals. Block sparse Bayesian learning (BSBL) is an effective approach to recover such signals with satisfactory recovery quality. However, it is time-consuming in recovering multichannel signals, since its computational load almost linearly increases with the number of channels. This work proposes a spatiotemporal sparse Bayesian learning algorithm to recover multichannel signals simultaneously. It not only exploits temporal correlation within each channel signal, but also exploits inter-channel correlation among different channel signals. Furthermore, its computational load is not significantly affected by the number of channels. The proposed algorithm was applied to brain computer interface (BCI) and EEG-based driver's drowsiness estimation. Results showed that the algorithm had both better recovery performance and much higher speed than BSBL. Particularly, the proposed algorithm ensured that the BCI classification and the drowsiness estimation had little degradation even when data were compressed by 80%, making it very suitable for continuous wireless telemonitoring of multichannel signals.

cs.IT↗

arXiv2014

Compressed Sensing of EEG for Wireless Telemonitoring with Low Energy Consumption and Inexpensive Hardware

Telemonitoring of electroencephalogram (EEG) through wireless body-area networks is an evolving direction in personalized medicine. Among various constraints in designing such a system, three important constraints are energy consumption, data compression, and device cost. Conventional data compression methodologies, although effective in data compression, consumes significant energy and cannot reduce device cost. Compressed sensing (CS), as an emerging data compression methodology, is promising in catering to these constraints. However, EEG is non-sparse in the time domain and also non-sparse in transformed domains (such as the wavelet domain). Therefore, it is extremely difficult for current CS algorithms to recover EEG with the quality that satisfies the requirements of clinical diagnosis and engineering applications. Recently, Block Sparse Bayesian Learning (BSBL) was proposed as a new method to the CS problem. This study introduces the technique to the telemonitoring of EEG. Experimental results show that its recovery quality is better than state-of-the-art CS algorithms, and sufficient for practical use. These results suggest that BSBL is very promising for telemonitoring of EEG and other non-sparse physiological signals.

stat.AP↗

arXiv2014

Compressed Sensing for Energy-Efficient Wireless Telemonitoring of Noninvasive Fetal ECG via Block Sparse Bayesian Learning

Fetal ECG (FECG) telemonitoring is an important branch in telemedicine. The design of a telemonitoring system via a wireless body-area network with low energy consumption for ambulatory use is highly desirable. As an emerging technique, compressed sensing (CS) shows great promise in compressing/reconstructing data with low energy consumption. However, due to some specific characteristics of raw FECG recordings such as non-sparsity and strong noise contamination, current CS algorithms generally fail in this application. This work proposes to use the block sparse Bayesian learning (BSBL) framework to compress/reconstruct non-sparse raw FECG recordings. Experimental results show that the framework can reconstruct the raw recordings with high quality. Especially, the reconstruction does not destroy the interdependence relation among the multichannel recordings. This ensures that the independent component analysis decomposition of the reconstructed recordings has high fidelity. Furthermore, the framework allows the use of a sparse binary sensing matrix with much fewer nonzero entries to compress recordings. Particularly, each column of the matrix can contain only two nonzero entries. This shows the framework, compared to other algorithms such as current CS algorithms and wavelet algorithms, can greatly reduce code execution in CPU in the data compression stage.

stat.ML↗

arXiv2014

Extension of SBL Algorithms for the Recovery of Block Sparse Signals with Intra-Block Correlation

We examine the recovery of block sparse signals and extend the framework in two important directions; one by exploiting signals' intra-block correlation and the other by generalizing signals' block structure. We propose two families of algorithms based on the framework of block sparse Bayesian learning (BSBL). One family, directly derived from the BSBL framework, requires knowledge of the block structure. Another family, derived from an expanded BSBL framework, is based on a weaker assumption on the block structure, and can be used when the block structure is completely unknown. Using these algorithms we show that exploiting intra-block correlation is very helpful in improving recovery performance. These algorithms also shed light on how to modify existing algorithms or design new ones to exploit such correlation and improve performance.

stat.ML↗

arXiv2014

Compressed Sensing for Energy-Efficient Wireless Telemonitoring: Challenges and Opportunities

As a lossy compression framework, compressed sensing has drawn much attention in wireless telemonitoring of biosignals due to its ability to reduce energy consumption and make possible the design of low-power devices. However, the non-sparseness of biosignals presents a major challenge to compressed sensing. This study proposes and evaluates a spatio-temporal sparse Bayesian learning algorithm, which has the desired ability to recover such non-sparse biosignals. It exploits both temporal correlation in each individual biosignal and inter-channel correlation among biosignals from different channels. The proposed algorithm was used for compressed sensing of multichannel electroencephalographic (EEG) signals for estimating vehicle drivers' drowsiness. Results showed that the drowsiness estimation was almost unaffected even if raw EEG signals (containing various artifacts) were compressed by 90%.

cs.IT↗

arXiv2013

Multiuser Diversity in Interfering Broadcast Channels: Achievable Degrees of Freedom and User Scaling Law

This paper investigates how multiuser dimensions can effectively be exploited for target degrees of freedom (DoF) in interfering broadcast channels (IBC) consisting of K-transmitters and their user groups. First, each transmitter is assumed to have a single antenna and serve a singe user in its user group where each user has receive antennas less than K. In this case, a K-transmitter single-input multiple-output (SIMO) interference channel (IC) is constituted after user selection. Without help of multiuser diversity, K-1 interfering signals cannot be perfectly removed at each user since the number of receive antennas is smaller than or equal to the number of interferers. Only with proper user selection, non-zero DoF per transmitter is achievable as the number of users increases. Through geometric interpretation of interfering channels, we show that the multiuser dimensions have to be used first for reducing the DoF loss caused by the interfering signals, and then have to be used for increasing the DoF gain from its own signal. The sufficient number of users for the target DoF is derived. We also discuss how the optimal strategy of exploiting multiuser diversity can be realized by practical user selection schemes. Finally, the single transmit antenna case is extended to the multiple-input multiple-output (MIMO) IBC where each transmitter with multiple antennas serves multiple users.

cs.IT↗

arXiv2013

Joint Beamforming and Power Control in Coordinated Multicell: Max-Min Duality, Effective Network and Large System Transition

This paper studies joint beamforming and power control in a coordinated multicell downlink system that serves multiple users per cell to maximize the minimum weighted signal-to-interference-plus-noise ratio. The optimal solution and distributed algorithm with geometrically fast convergence rate are derived by employing the nonlinear Perron-Frobenius theory and the multicell network duality. The iterative algorithm, though operating in a distributed manner, still requires instantaneous power update within the coordinated cluster through the backhaul. The backhaul information exchange and message passing may become prohibitive with increasing number of transmit antennas and increasing number of users. In order to derive asymptotically optimal solution, random matrix theory is leveraged to design a distributed algorithm that only requires statistical information. The advantage of our approach is that there is no instantaneous power update through backhaul. Moreover, by using nonlinear Perron-Frobenius theory and random matrix theory, an effective primal network and an effective dual network are proposed to characterize and interpret the asymptotic solution.

cs.IT↗

arXiv2013

Multicell Random Beamforming with CDF-based Scheduling: Exact Rate and Scaling Laws

In a multicell multiuser MIMO downlink employing random beamforming as the transmission scheme, the heterogeneous large scale channel effects of intercell and intracell interference complicate analysis of distributed scheduling based systems. In this paper, we extend the analysis in [1] and [2] to study the aforementioned challenging scenario. The cumulative distribution function (CDF)-based scheduling policy utilized in [1] and [2] is leveraged to maintain fairness among users and simultaneously obtain multiuser diversity gain. The closed form expression of the individual sum rate for each user is derived under the CDF-based scheduling policy. More importantly, with this distributed scheduling policy, we conduct asymptotic (in users) analysis to determine the limiting distribution of the signal-to-interference-plus-noise ratio, and establish the individual scaling laws for each user.

cs.IT↗

arXiv2013

Random Beamforming with Heterogeneous Users and Selective Feedback: Individual Sum Rate and Individual Scaling Laws

This paper investigates three open problems in random beamforming based communication systems: the scheduling policy with heterogeneous users, the closed form sum rate, and the randomness of multiuser diversity with selective feedback. By employing the cumulative distribution function based scheduling policy, we guarantee fairness among users as well as obtain multiuser diversity gain in the heterogeneous scenario. Under this scheduling framework, the individual sum rate, namely the average rate for a given user multiplied by the number of users, is of interest and analyzed under different feedback schemes. Firstly, under the full feedback scheme, we derive the closed form individual sum rate by employing a decomposition of the probability density function of the selected user's signal-to-interference-plus-noise ratio. This technique is employed to further obtain a closed form rate approximation with selective feedback in the spatial dimension. The analysis is also extended to random beamforming in a wideband OFDMA system with additional selective feedback in the spectral dimension wherein only the best beams for the best-L resource blocks are fed back. We utilize extreme value theory to examine the randomness of multiuser diversity incurred by selective feedback. Finally, by leveraging the tail equivalence method, the multiplicative effect of selective feedback and random observations is observed to establish the individual rate scaling.

cs.IT↗