arXiv subjects

Babak Hassibi

Publications and source records attributed to Babak Hassibi.

At least 91 records · Page 5Linked to original sources

arXiv2018

Noncoherent Short-Packet Communication via Modulation on Conjugated Zeros

We introduce a novel blind (noncoherent) communication scheme, called modulation on conjugate-reciprocal zeros (MOCZ), to reliably transmit short binary packets over unknown finite impulse response systems as used, for example, to model underspread wireless multipath channels. In MOCZ, the information is modulated onto the zeros of the transmitted signals $z-$transform. In the absence of additive noise, the zero structure of the signal is perfectly preserved at the receiver, no matter what the channel impulse response (CIR) is. Furthermore, by a proper selection of the zeros, we show that MOCZ is not only invariant to the CIR, but also robust against additive noise. Starting with the maximum-likelihood estimator, we define a low complexity and reliable decoder and compare it to various state-of-the art noncoherent schemes.

cs.IT↗

arXiv2018

Optimum Linear Codes with Support Constraints over Small Fields

We consider the problem of designing optimal linear codes (in terms of having the largest minimum distance) subject to a support constraint on the generator matrix. We show that the largest minimum distance can be achieved by a subcode of a Reed-Solomon code of small field size. As a by-product of this result, we settle the GM-MDS conjecture of Dau et. al. in the affirmative.

cs.IT↗

arXiv2018

Manifold Optimization Over the Set of Doubly Stochastic Matrices: A Second-Order Geometry

Convex optimization is a well-established research area with applications in almost all fields. Over the decades, multiple approaches have been proposed to solve convex programs. The development of interior-point methods allowed solving a more general set of convex programs known as semi-definite programs and second-order cone programs. However, it has been established that these methods are excessively slow for high dimensions, i.e., they suffer from the curse of dimensionality. On the other hand, optimization algorithms on manifold have shown great ability in finding solutions to nonconvex problems in reasonable time. This paper is interested in solving a subset of convex optimization using a different approach. The main idea behind Riemannian optimization is to view the constrained optimization problem as an unconstrained one over a restricted search space. The paper introduces three manifolds to solve convex programs under particular box constraints. The manifolds, called the doubly stochastic, symmetric and the definite multinomial manifolds, generalize the simplex also known as the multinomial manifold. The proposed manifolds and algorithms are well-adapted to solving convex programs in which the variable of interest is a multidimensional probability distribution function. Theoretical analysis and simulation results testify the efficiency of the proposed method over state of the art methods. In particular, they reveal that the proposed framework outperforms conventional generic and specialized solvers, especially in high dimensions.

math.OC↗

arXiv2018

Further Progress on the GM-MDS Conjecture for Reed-Solomon Codes

Designing good error correcting codes whose generator matrix has a support constraint, i.e., one for which only certain entries of the generator matrix are allowed to be non-zero, has found many recent applications, including in distributed coding and storage, multiple access networks, and weakly secure data exchange. The dual problem, where the parity check matrix has a support constraint, comes up in the design of locally repairable codes. The central problem here is to design codes with the largest possible minimum distance, subject to the given support constraint on the generator matrix. An upper bound on the minimum distance can be obtained through a set of singleton bounds, which can be alternatively thought of as a cut-set bound. Furthermore, it is well known that, if the field size is large enough, any random generator matrix obeying the support constraint will achieve the maximum minimum distance with high probability. Since random codes are not easy to decode, structured codes with efficient decoders, e.g., Reed-Solomon codes, are much more desirable. The GM-MDS conjecture of Dau et al states that the maximum minimum distance over all codes satisfying the generator matrix support constraint can be obtained by a Reed Solomon code. If true, this would have significant consequences. The conjecture has been proven for several special case: when the dimension of the code k is less than or equal to five, when the number of distinct support sets on the rows of the generator matrix m, say, is less than or equal to three, or when the generator matrix is sparsest and balanced. In this paper, we report on further progress on the GM-MDS conjecture. In particular, we show that the conjecture is true for all m less than equal to six. This generalizes all previous known results (except for the sparsest and balanced case, which is a very special support constraint).

cs.IT↗

arXiv2018

A Precise Analysis of PhaseMax in Phase Retrieval

Recovering an unknown complex signal from the magnitude of linear combinations of the signal is referred to as phase retrieval. We present an exact performance analysis of a recently proposed convex-optimization-formulation for this problem, known as PhaseMax. Standard convex-relaxation-based methods in phase retrieval resort to the idea of "lifting" which makes them computationally inefficient, since the number of unknowns is effectively squared. In contrast, PhaseMax is a novel convex relaxation that does not increase the number of unknowns. Instead it relies on an initial estimate of the true signal which must be externally provided. In this paper, we investigate the required number of measurements for exact recovery of the signal in the large system limit and when the linear measurement matrix is random with iid standard normal entries. If $n$ denotes the dimension of the unknown complex signal and $m$ the number of phaseless measurements, then in the large system limit, $\frac{m}{n} > \frac{4}{\cos^2(θ)}$ measurements is necessary and sufficient to recover the signal with high probability, where $θ$ is the angle between the initial estimate and the true signal. Our result indicates a sharp phase transition in the asymptotic regime which matches the empirical result in numerical simulations.

cs.IT↗

arXiv2017

Distributed Solution of Large-Scale Linear Systems via Accelerated Projection-Based Consensus

Solving a large-scale system of linear equations is a key step at the heart of many algorithms in machine learning, scientific computing, and beyond. When the problem dimension is large, computational and/or memory constraints make it desirable, or even necessary, to perform the task in a distributed fashion. In this paper, we consider a common scenario in which a taskmaster intends to solve a large-scale system of linear equations by distributing subsets of the equations among a number of computing machines/cores. We propose an accelerated distributed consensus algorithm, in which at each iteration every machine updates its solution by adding a scaled version of the projection of an error signal onto the nullspace of its system of equations, and where the taskmaster conducts an averaging over the solutions with momentum. The convergence behavior of the proposed algorithm is analyzed in detail and analytically shown to compare favorably with the convergence rate of alternative distributed methods, namely distributed gradient descent, distributed versions of Nesterov's accelerated gradient descent and heavy-ball method, the block Cimmino method, and ADMM. On randomly chosen linear systems, as well as on real-world data sets, the proposed method offers significant speed-up relative to all the aforementioned methods. Finally, our analysis suggests a novel variation of the distributed heavy-ball method, which employs a particular distributed preconditioning, and which achieves the same theoretical convergence rate as the proposed consensus-based method.

cs.LG↗

arXiv2017

Symbol Error Rate Performance of Box-relaxation Decoders in Massive MIMO

The maximum-likelihood (ML) decoder for symbol detection in large multiple-input multiple-output wireless communication systems is typically computationally prohibitive. In this paper, we study a popular and practical alternative, namely the Box-relaxation optimization (BRO) decoder, which is a natural convex relaxation of the ML. For iid real Gaussian channels with additive Gaussian noise, we obtain exact asymptotic expressions for the symbol error rate (SER) of the BRO. The formulas are particularly simple, they yield useful insights, and they allow accurate comparisons to the matched-filter bound (MFB) and to the zero-forcing decoder. For BPSK signals the SER performance of the BRO is within 3dB of the MFB for square systems, and it approaches the MFB as the number of receive antennas grows large compared to the number of transmit antennas. Our analysis further characterizes the empirical density function of the solution of the BRO, and shows that error events for any fixed number of symbols are asymptotically independent. The fundamental tool behind the analysis is the convex Gaussian min-max theorem.

eess.SP↗

arXiv2017

Stable Blind Deconvolution over the Reals from Additional Autocorrelations

Recently the one-dimensional time-discrete blind deconvolution problem was shown to be solvable uniquely, up to a global phase, by a semi-definite program for almost any signal, provided its autocorrelation is known. We will show in this work that under a sufficient zero separation of the corresponding signal in the $z-$domain, a stable reconstruction against additive noise is possible. Moreover, the stability constant depends on the signal dimension and on the signals magnitude of the first and last coefficients. We give an analytical expression for this constant by using spectral bounds of Vandermonde matrices.

cs.IT↗

arXiv2017

Rate-cost tradeoffs in control. Part II: achievable scheme

Consider a distributed control problem with a communication channel connecting the observer of a linear stochastic system to the controller. The goal of the controller is to minimize a quadratic cost function in the state variables and control signal, known as the linear quadratic regulator (LQR). We study the fundamental tradeoff between the communication rate r bits/sec and the limsup of the expected cost b. In the companion paper, which can be read independently of the current one, we show a lower bound on a certain cost function, which quantifies the minimum mutual information between the channel input and output, given the past, that is compatible with a target LQR cost. The bound applies as long as the system noise has a probability density function, and it holds for a general class of codes that can take full advantage of the memory of the data observed so far and that are not constrained to have any particular structure. In this paper, we prove that the bound can be approached by a simple variable-length lattice quantization scheme, as long as the system noise satisfies a smoothness condition. The quantization scheme only quantizes the innovation, that is, the difference between the controller's belief about the current state and the encoder's state estimate. Our proof technique leverages some recent results on nonasymptotic high resolution vector quantization.

cs.IT↗

arXiv2017

Improving Distributed Gradient Descent Using Reed-Solomon Codes

Today's massively-sized datasets have made it necessary to often perform computations on them in a distributed manner. In principle, a computational task is divided into subtasks which are distributed over a cluster operated by a taskmaster. One issue faced in practice is the delay incurred due to the presence of slow machines, known as \emph{stragglers}. Several schemes, including those based on replication, have been proposed in the literature to mitigate the effects of stragglers and more recently, those inspired by coding theory have begun to gain traction. In this work, we consider a distributed gradient descent setting suitable for a wide class of machine learning problems. We adapt the framework of Tandon et al. (arXiv:1612.03301) and present a deterministic scheme that, for a prescribed per-machine computational effort, recovers the gradient from the least number of machines $f$ theoretically permissible, via an $O(f^2)$ decoding algorithm. We also provide a theoretical delay model which can be used to minimize the expected waiting time per computation by optimally choosing the parameters of the scheme. Finally, we supplement our theoretical findings with numerical results that demonstrate the efficacy of the method and its advantages over competing schemes.

cs.IT↗

arXiv2017

Short-Message Communication and FIR System Identification using Huffman Sequences

Providing short-message communication and simultaneous channel estimation for sporadic and fast fading scenarios is a challenge for future wireless networks. In this work we propose a novel blind communication and deconvolution scheme by using Huffman sequences, which allows to solve three important tasks in one step: (i) determination of the transmit power (ii) identification of the discrete-time FIR channel by providing a maximum delay of less than $L/2$ and (iii) simultaneously communicating $L-1$ bits of information. Our signal reconstruction uses a recent semi-definite program that can recover two unknown signals from their auto-correlations and cross-correlations. This convex algorithm is stable and operates fully deterministic without any further channel assumptions.

cs.IT↗

arXiv2017

Entropic Causality and Greedy Minimum Entropy Coupling

We study the problem of identifying the causal relationship between two discrete random variables from observational data. We recently proposed a novel framework called entropic causality that works in a very general functional model but makes the assumption that the unobserved exogenous variable has small entropy in the true causal direction. This framework requires the solution of a minimum entropy coupling problem: Given marginal distributions of m discrete random variables, each on n states, find the joint distribution with minimum entropy, that respects the given marginals. This corresponds to minimizing a concave function of nm variables over a convex polytope defined by nm linear constraints, called a transportation polytope. Unfortunately, it was recently shown that this minimum entropy coupling problem is NP-hard, even for 2 variables with n states. Even representing points (joint distributions) over this space can require exponential complexity (in n, m) if done naively. In our recent work we introduced an efficient greedy algorithm to find an approximate solution for this problem. In this paper we analyze this algorithm and establish two results: that our algorithm always finds a local minimum and also is within an additive approximation error from the unknown global optimum.

cs.IT↗

arXiv2017

Blind Deconvolution with Additional Autocorrelations via Convex Programs

In this work we characterize all ambiguities of the linear (aperiodic) one-dimensional convolution on two fixed finite-dimensional complex vector spaces. It will be shown that the convolution ambiguities can be mapped one-to-one to factorization ambiguities in the $z-$domain, which are generated by swapping the zeros of the input signals. We use this polynomial description to show a deterministic version of a recently introduced masked Fourier phase retrieval design. In the noise-free case a (convex) semi-definite program can be used to recover exactly the input signals if they share no common factors (zeros). Then, we reformulate the problem as deterministic blind deconvolution with prior knowledge of the autocorrelations. Numerically simulations show that our approach is also robust against additive noise.

cs.IT↗

arXiv2017

Multiple Illumination Phaseless Super-Resolution (MIPS) with Applications To Phaseless DOA Estimation and Diffraction Imaging

Phaseless super-resolution is the problem of recovering an unknown signal from measurements of the magnitudes of the low frequency Fourier transform of the signal. This problem arises in applications where measuring the phase, and making high-frequency measurements, are either too costly or altogether infeasible. The problem is especially challenging because it combines the difficult problems of phase retrieval and classical super-resolution

cs.IT↗

arXiv2016

Entropic Causal Inference

We consider the problem of identifying the causal direction between two discrete random variables using observational data. Unlike previous work, we keep the most general functional model but make an assumption on the unobserved exogenous variable: Inspired by Occam's razor, we assume that the exogenous variable is simple in the true causal direction. We quantify simplicity using Rényi entropy. Our main result is that, under natural assumptions, if the exogenous variable has low $H_0$ entropy (cardinality) in the true direction, it must have high $H_0$ entropy in the wrong direction. We establish several algorithmic hardness results about estimating the minimum entropy exogenous variable. We show that the problem of finding the exogenous variable with minimum entropy is equivalent to the problem of finding minimum joint entropy given $n$ marginal distributions, also known as minimum entropy coupling problem. We propose an efficient greedy algorithm for the minimum entropy coupling problem, that for $n=2$ provably finds a local optimum. This gives a greedy algorithm for finding the exogenous variable with minimum $H_1$ (Shannon Entropy). Our greedy entropy-based causal inference algorithm has similar performance to the state of the art additive noise models in real datasets. One advantage of our approach is that we make no use of the values of random variables but only their distributions. Our method can therefore be used for causal inference for both ordinal and also categorical data, unlike additive noise models.

cs.AI↗

arXiv2016

Multi-Rate Control over AWGN Channels via Analog Joint Source-Channel Coding

We consider the problem of controlling an unstable plant over an additive white Gaussian noise (AWGN) channel with a transmit power constraint, where the signaling rate of communication is larger than the sampling rate (for generating observations and applying control inputs) of the underlying plant. Such a situation is quite common since sampling is done at a rate that captures the dynamics of the plant and which is often much lower than the rate that can be communicated. This setting offers the opportunity of improving the system performance by employing multiple channel uses to convey a single message (output plant observation or control input). Common ways of doing so are through either repeating the message, or by quantizing it to a number of bits and then transmitting a channel coded version of the bits whose length is commensurate with the number of channel uses per sampled message. We argue that such "separated source and channel coding" can be suboptimal and propose to perform joint source-channel coding. Since the block length is short we obviate the need to go to the digital domain altogether and instead consider analog joint source-channel coding. For the case where the communication signaling rate is twice the sampling rate, we employ the Archimedean bi-spiral-based Shannon-Kotel'nikov analog maps to show significant improvement in stability margins and linear-quadratic Gaussian (LQG) costs over simple schemes that employ repetition.

eess.SY↗

arXiv2016

Reconstruction of signals from their autocorrelation and cross-correlation vectors, with applications to phase retrieval and blind channel estimation

We consider the problem of reconstructing two signals from the autocorrelation and cross-correlation measurements. This inverse problem is a fundamental one in signal processing, and arises in many applications, including phase retrieval and blind channel estimation. In a typical phase retrieval setup, only the autocorrelation measurements are obtainable. We show that, when the measurements are obtained using three simple "masks", phase retrieval reduces to the aforementioned reconstruction problem. The classic solution to this problem is based on finding common factors between the $z$-transforms of the autocorrelation and cross-correlation vectors. This solution has enjoyed limited practical success, mainly due to the fact that it is not sufficiently stable in the noisy setting. In this work, inspired by the success of convex programming in provably and stably solving various quadratic constrained problems, we develop a semidefinite programming-based algorithm and provide theoretical guarantees. In particular, we show that almost all signals can be uniquely recovered by this algorithm (up to a global phase). Comparative numerical studies demonstrate that the proposed method significantly outperforms the classic method in the noisy setting.

cs.IT↗

arXiv2016

Analysis of Exact and Approximated Epidemic Models over Complex Networks

We study the spread of discrete-time epidemics over arbitrary networks for well-known propagation models, namely SIS (susceptible-infected-susceptible), SIR (susceptible-infected-recovered), SIRS (susceptible-infected-recovered-susceptible) and SIV (susceptible-infected-vaccinated). Such epidemics are described by $2^n$- or $3^n$-state Markov chains. Ostensibly, because analyzing such Markov chains is too complicated, their $O(n)$-dimensional nonlinear "mean-field" approximation, and its linearization, are often studied instead. We provide a complete global analysis of the epidemic dynamics of the nonlinear mean-field approximation. In particular, we show that depending on the largest eigenvalue of the underlying graph adjacency matrix and the rates of infection, recovery, and vaccination, the global dynamics takes on one of two forms: either the epidemic dies out, or it converges to another unique fixed point (the so-called endemic state where a constant fraction of the nodes remain infected). A similar result has also been shown in the continuous-time case. We tie in these results with the "true" underlying Markov chain model by showing that the linear model is the tightest upper-bound on the true probabilities of infection that involves only marginals, and that, even though the nonlinear model is not an upper-bound on the true probabilities in general, it does provide an upper-bound on the probability of the chain not being absorbed. As a consequence, we also show that when the disease-free fixed point is globally stable for the mean-field model, the Markov chain has an $O(\log n)$ mixing time, which means the epidemic dies out quickly. We compare and summarize the results on different propagation models.

cs.SI↗