arXiv subjects

Krishna Narayanan

Publications and source records attributed to Krishna Narayanan.

35 records · Page 2Linked to original sources

arXiv2023

On the Advantages of Asynchrony in the Unsourced MAC

In this work we demonstrate how a lack of synchronization can in fact be advantageous in the problem of random access. Specifically, we consider a multiple-access problem over a frame-asynchronous 2-user binary-input adder channel in the unsourced setup (2-UBAC). Previous work has shown that under perfect synchronization the per-user rates achievable with linear codes over the 2-UBAC are limited by 0.5 bit per channel use (compared to the capacity of 0.75). In this paper, we first demonstrate that arbitrary small (even single-bit) shift between the user's frames enables (random) linear codes to attain full capacity of 0.75 bit/user. Furthermore, we derive density evolution equations for irregular LDPC codes, and prove (via concentration arguments) that they correctly track the asymptotic bit-error rate of a BP decoder. Optimizing the degree distributions we construct LDPC codes achieving per-user rates of 0.73 bit per channel use.

cs.IT↗

arXiv2022

Bayesian Graph Contrastive Learning

Contrastive learning has become a key component of self-supervised learning approaches for graph-structured data. Despite their success, existing graph contrastive learning methods are incapable of uncertainty quantification for node representations or their downstream tasks, limiting their application in high-stakes domains. In this paper, we propose a novel Bayesian perspective of graph contrastive learning methods showing random augmentations leads to stochastic encoders. As a result, our proposed method represents each node by a distribution in the latent space in contrast to existing techniques which embed each node to a deterministic vector. By learning distributional representations, we provide uncertainty estimates in downstream graph analytics tasks and increase the expressive power of the predictive model. In addition, we propose a Bayesian framework to infer the probability of perturbations in each view of the contrastive model, eliminating the need for a computationally expensive search for hyperparameter tuning. We empirically show a considerable improvement in performance compared to existing state-of-the-art methods on several benchmark datasets.

cs.LG↗

arXiv2022

On Linear Separability under Linear Compression with Applications to Hard Support Vector Machine

This paper investigates the theoretical problem of maintaining linear separability of the data-generating distribution under linear compression. While it has been long known that linear separability may be maintained by linear transformations that approximately preserve the inner products between the domain points, the limit to which the inner products are preserved in order to maintain linear separability was unknown. In this paper, we show that linear separability is maintained as long as the distortion of the inner products is smaller than the squared margin of the original data-generating distribution. The proof is mainly based on the geometry of hard support vector machines (SVM) extended from the finite set of training examples to the (possibly) infinite domain of the data-generating distribution. As applications, we derive bounds on the (i) compression length of random sub-Gaussian matrices; and (ii) generalization error for compressive learning with hard-SVM.

cs.LG↗

arXiv2021

Approximate Support Recovery using Codes for Unsourced Multiple Access

We consider the approximate support recovery (ASR) task of inferring the support of a $K$-sparse vector ${\bf x} \in \mathbb{R}^n$ from $m$ noisy measurements. We examine the case where $n$ is large, which precludes the application of standard compressed sensing solvers, thereby necessitating solutions with lower complexity. We design a scheme for ASR by leveraging techniques developed for unsourced multiple access. We present two decoding algorithms with computational complexities $\mathcal{O}(K^2 \log n+K \log n \log \log n)$ and $\mathcal{O}(K^3 +K^2 \log n+ K \log n \log \log n)$ per iteration, respectively. When $K \ll n$, this is much lower than the complexity of approximate message passing with a minimum mean squared error denoiser% (AMP-MMSE) ,which requires $\mathcal{O}(mn)$ operations per iteration. This gain comes at a slight performance cost. Our findings suggest that notions from multiple access %such as spreading, matched filter receivers and codes can play an important role in the design of measurement schemes for ASR.

cs.IT↗

arXiv2020

Bayesian Graph Neural Networks with Adaptive Connection Sampling

We propose a unified framework for adaptive connection sampling in graph neural networks (GNNs) that generalizes existing stochastic regularization methods for training GNNs. The proposed framework not only alleviates over-smoothing and over-fitting tendencies of deep GNNs, but also enables learning with uncertainty in graph analytic tasks with GNNs. Instead of using fixed sampling rates or hand-tuning them as model hyperparameters in existing stochastic regularization methods, our adaptive connection sampling can be trained jointly with GNN model parameters in both global and local fashions. GNN training with adaptive connection sampling is shown to be mathematically equivalent to an efficient approximation of training Bayesian GNNs. Experimental results with ablation studies on benchmark datasets validate that adaptively learning the sampling rate given graph training data is the key to boost the performance of GNNs in semi-supervised node classification, less prone to over-smoothing and over-fitting with more robust prediction.

cs.LG↗

arXiv2020

Semi-Implicit Stochastic Recurrent Neural Networks

Stochastic recurrent neural networks with latent random variables of complex dependency structures have shown to be more successful in modeling sequential data than deterministic deep models. However, the majority of existing methods have limited expressive power due to the Gaussian assumption of latent variables. In this paper, we advocate learning implicit latent representations using semi-implicit variational inference to further increase model flexibility. Semi-implicit stochastic recurrent neural network(SIS-RNN) is developed to enrich inferred model posteriors that may have no analytic density functions, as long as independent random samples can be generated via reparameterization. Extensive experiments in different tasks on real-world datasets show that SIS-RNN outperforms the existing methods.

cs.LG↗

arXiv2018

Rate Selection and Power Adaptation using Maximal Ratio Combining for the Random Access Gaussian Channel

With the emergence of machine-driven communi- cation, there is a renewed interest in the design of random multiple access schemes for networks with large number of active devices. Many of the recently proposed access paradigms are enhancements to slotted ALOHA. One of the popular schemes, irregular repetition slotted ALOHA (IRSA), is based on an analogy between multiple access with successive interference cancellation and message-passing decoding on bipartite graphs. Most of the results on IRSA and its variants focus on the collision channel and they ignore physical limitations such as transmit power constraints and additive Gaussian noise at the physical layer. As such, naive extensions of IRSA to the Gaussian multiple access channel are not power efficient in the low signal-to-noise- ratio regime. This work introduces a novel paradigm whereby devices adapt their rates and/or transmit powers based on their chosen repetition profiles. The receiver performs maximal ratio combining over all the replicas prior to decoding a message. Numerical results for finite number of users show that the proposed scheme can provide substantial improvements in terms of power efficiency and average rate over standard IRSA.

cs.IT↗

arXiv2012

On Modulo-Sum Computation over an Erasure Multiple Access Channel

We study computation of a modulo-sum of two binary source sequences over a two-user erasure multiple access channel. The channel is modeled as a binary-input, erasure multiple access channel, which can be in one of three states - either the channel output is a modulo-sum of the two input symbols, or the channel output equals the input symbol on the first link and an erasure on the second link, or vice versa. The associated state sequence is independent and identically distributed. We develop a new upper bound on the sum-rate by revealing only part of the state sequence to the transmitters. Our coding scheme is based on the compute and forward and the decode and forward techniques. When a (strictly) causal feedback of the channel state is available to the encoders, we show that the modulo-sum capacity is increased. Extensions to the case of lossy reconstruction of the modulo-sum and to channels involving additional states are also treated briefly.

cs.IT↗

arXiv2012

Joint Compute and Forward for the Two Way Relay Channel with Spatially Coupled LDPC Codes

We consider the design and analysis of coding schemes for the binary input two way relay channel with erasure noise. We are particularly interested in reliable physical layer network coding in which the relay performs perfect error correction prior to forwarding messages. The best known achievable rates for this problem can be achieved through either decode and forward or compute and forward relaying. We consider a decoding paradigm called joint compute and forward which we numerically show can achieve the best of these rates with a single encoder and decoder. This is accomplished by deriving the exact performance of a message passing decoder based on joint compute and forward for spatially coupled LDPC ensembles.

cs.IT↗

arXiv2011

Multilevel Coding Schemes for Compute-and-Forward with Flexible Decoding

We consider the design of coding schemes for the wireless two-way relaying channel when there is no channel state information at the transmitter. In the spirit of the compute and forward paradigm, we present a multilevel coding scheme that permits computation (or, decoding) of a class of functions at the relay. The function to be computed (or, decoded) is then chosen depending on the channel realization. We define such a class of functions which can be decoded at the relay using the proposed coding scheme and derive rates that are universally achievable over a set of channel gains when this class of functions is used at the relay. We develop our framework with general modulation formats in mind, but numerical results are presented for the case where each node transmits using the QPSK constellation. Numerical results with QPSK show that the flexibility afforded by our proposed scheme results in substantially higher rates than those achievable by always using a fixed function or by adapting the function at the relay but coding over GF(4).

cs.IT↗

arXiv2011

Multilevel Coding Schemes for Compute-and-Forward

We investigate techniques for designing modulation/coding schemes for the wireless two-way relaying channel. The relay is assumed to have perfect channel state information, but the transmitters are assumed to have no channel state information. We consider physical layer network coding based on multilevel coding techniques. Our multilevel coding framework is inspired by the compute-and-forward relaying protocol. Indeed, we show that the framework developed here naturally facilitates decoding of linear combinations of codewords for forwarding by the relay node. We develop our framework with general modulation formats in mind, but numerical results are presented for the case where each node transmits using the QPSK constellation with gray labeling. We focus our discussion on the rates at which the relay may reliably decode linear combinations of codewords transmitted from the end nodes.

cs.IT↗

arXiv2011

Universality for the Noisy Slepian-Wolf Problem Via Spatial Coupling

We consider a noisy Slepian-Wolf problem where two correlated sources are separately encoded and transmitted over two independent binary memoryless symmetric channels. Each channel capacity is assumed to be characterized by a single parameter which is not known at the transmitter. The receiver has knowledge of both the source correlation and the channel parameters. We call a system universal if it retains near-capacity performance without channel knowledge at the transmitter. Kudekar et al. recently showed that terminated low-density parity-check (LDPC) convolutional codes (a.k.a. spatially-coupled LDPC ensembles) can have belief-propagation thresholds that approach their maximum a-posteriori thresholds. This was proven for binary erasure channels and shown empirically for binary memoryless symmetric channels. They also conjectured that the principle of spatial coupling is very general and the phenomenon of threshold saturation applies to a very broad class of graphical models. In this work, we derive an area theorem for the joint decoder and empirically show that threshold saturation occurs for this problem. As a result, we demonstrate near-universal performance for this problem using the proposed spatially-coupled coding system. A similar result is also discussed briefly for the 2-user multiple-access channel.

cs.IT↗

arXiv2009

Power Allocation Strategies and Lattice Based Coding schemes for Bi-directional relaying

We consider a communication system where two transmitters wish to exchange information through a half-duplex relay in the middle. The channels between the transmitters and the relay have asymmetric channel gains. More specifically, the channels are assumed to be synchronized with complex inputs and complex fading coefficients with an average power constraint on the inputs to the channels. The noise at the receivers have the same power spectral density and are assumed to be white and Gaussian. We restrict our attention to transmission schemes where information from the two nodes are simultaneously sent to the relay during a medium access phase followed by a broadcast phase where the relay broadcasts information to both the nodes. An upper bound on the capacity for the two phase protocol under a sum power constraint on the transmit power from all the nodes is obtained as a solution to a convex optimization problem. We show that a scheme using channel inversion with lattice decoding can obtain a rate a small constant 0.09 bits from the upper bound at high signal-to-noise ratios. Numerical results show that the proposed scheme can perform very close to the upper bound.

cs.IT↗

arXiv2009

Transmitting an analog Gaussian source over a Gaussian wiretap channel under SNR mismatch

In this work we study encoding/decoding schemes for the transmission of a discrete time analog Gaussian source over a Gaussian wiretap channel. The intended receiver is assumed to have a certain minimum signal to noise ratio (SNR) and the eavesdropper is assumed to have a strictly lower SNR compared to the intended receiver. For a fixed information leakage to the eavesdropper, we are interested in minimizing the distortion in source reconstruction at the intended receiver, and we propose joint source channel coding (JSCC) schemes for this setup. For a fixed information leakage to the eavesdropper, we also show that the schemes considered give a graceful degradation of distortion with SNR under SNR mismatch, i.e., when the actual channel SNR is observed to be different from the design SNR.

cs.IT↗

arXiv2008

Joint Physical Layer Coding and Network Coding for Bi-Directional Relaying

We consider the problem of two transmitters wishing to exchange information through a relay in the middle. The channels between the transmitters and the relay are assumed to be synchronized, average power constrained additive white Gaussian noise channels with a real input with signal-to-noise ratio (SNR) of snr. An upper bound on the capacity is 1/2 log(1+ snr) bits per transmitter per use of the medium-access phase and broadcast phase of the bi-directional relay channel. We show that using lattice codes and lattice decoding, we can obtain a rate of 1/2 log(0.5 + snr) bits per transmitter, which is essentially optimal at high SNRs. The main idea is to decode the sum of the codewords modulo a lattice at the relay followed by a broadcast phase which performs Slepian-Wolf coding with structured codes. For asymptotically low SNR's, jointly decoding the two transmissions at the relay (MAC channel) is shown to be optimal. We also show that if the two transmitters use identical lattices with minimum angle decoding, we can achieve the same rate of 1/2 log(0.5 + snr). The proposed scheme can be thought of as a joint physical layer, network layer code which outperforms other recently proposed analog network coding schemes.

cs.IT↗

arXiv2008

Joint Source Channel Coding with Side Information Using Hybrid Digital Analog Codes

We study the joint source channel coding problem of transmitting an analog source over a Gaussian channel in two cases - (i) the presence of interference known only to the transmitter and (ii) in the presence of side information known only to the receiver. We introduce hybrid digital analog forms of the Costa and Wyner-Ziv coding schemes. Our schemes are based on random coding arguments and are different from the nested lattice schemes by Kochman and Zamir that use dithered quantization. We also discuss superimposed digital and analog schemes for the above problems which show that there are infinitely many schemes for achieving the optimal distortion for these problems. This provides an extension of the schemes by Bross et al to the interference/side information case. We then discuss applications of the hybrid digital analog schemes for transmitting under a channel signal-to-noise ratio mismatch and for broadcasting a Gaussian source with bandwidth compression.

cs.IT↗

arXiv2005

An MSE Based Ttransfer Chart to Analyze Iterative Decoding Schemes

An alternative to extrinsic information transfer (EXIT) charts called mean squared error (MSE) charts that use a measure related to the MSE instead of mutual information is proposed. Using the relationship between mutual information and minimum mean squared error (MMSE), a relationship between the rate of any code and the area under a plot of MSE versus signal to noise ratio (SNR) is obtained, when the log likelihood ratios (LLR) can be assumed to be from a Gaussian channel. Using this result, a theoretical justification is provided for designing concatenated codes by matching the EXIT charts of the inner and outer decoders, when the LLRs are Gaussian which is typically assumed for code design using EXIT charts. Finally, for the special case of AWGN channel it is shown that any capacity achieving code has an EXIT curve that is flat. This extends Ashikhmin et als results for erasure channels to the Gaussian channel.

cs.IT↗