arXiv subjects

Sarah J. Johnson

Publications and source records attributed to Sarah J. Johnson.

At least 19 recordsLinked to original sources

arXiv2024

Divide, Conquer, Combine Bayesian Decision Tree Sampling

Decision trees are commonly used predictive models due to their flexibility and interpretability. This paper is directed at quantifying the uncertainty of decision tree predictions by employing a Bayesian inference approach. This is challenging because these approaches need to explore both the tree structure space and the space of decision parameters associated with each tree structure. This has been handled by using Markov Chain Monte Carlo (MCMC) methods, where a Markov Chain is constructed to provide samples from the desired Bayesian estimate. Importantly, the structure and the decision parameters are tightly coupled; small changes in the tree structure can demand vastly different decision parameters to provide accurate predictions. A challenge for existing MCMC approaches is proposing joint changes in both the tree structure and the decision parameters that result in efficient sampling. This paper takes a different approach, where each distinct tree structure is associated with a unique set of decision parameters. The proposed approach, entitled DCC-Tree, is inspired by the work in Zhou et al. [23] for probabilistic programs and Cochrane et al. [4] for Hamiltonian Monte Carlo (HMC) based sampling for decision trees. Results show that DCC-Tree performs comparably to other HMC-based methods and better than existing Bayesian tree methods while improving on consistency and reducing the per-proposal complexity.

cs.LG↗

arXiv2023

RJHMC-Tree for Exploration of the Bayesian Decision Tree Posterior

Decision trees have found widespread application within the machine learning community due to their flexibility and interpretability. This paper is directed towards learning decision trees from data using a Bayesian approach, which is challenging due to the potentially enormous parameter space required to span all tree models. Several approaches have been proposed to combat this challenge, with one of the more successful being Markov chain Monte Carlo (MCMC) methods. The efficacy and efficiency of MCMC methods fundamentally rely on the quality of the so-called proposals, which is the focus of this paper. In particular, this paper investigates using a Hamiltonian Monte Carlo (HMC) approach to explore the posterior of Bayesian decision trees more efficiently by exploiting the geometry of the likelihood within a global update scheme. Two implementations of the novel algorithm are developed and compared to existing methods by testing against standard datasets in the machine learning and Bayesian decision tree literature. HMC-based methods are shown to perform favourably with respect to predictive test accuracy, acceptance rate, and tree complexity.

cs.LG↗

arXiv2023

Joint User and Data Detection in Grant-Free NOMA with Attention-based BiLSTM Network

We consider the multi-user detection (MUD) problem in uplink grant-free non-orthogonal multiple access (NOMA), where the access point has to identify the total number and correct identity of the active Internet of Things (IoT) devices and decode their transmitted data. We assume that IoT devices use complex spreading sequences and transmit information in a random-access manner following the burst-sparsity model, where some IoT devices transmit their data in multiple adjacent time slots with a high probability, while others transmit only once during a frame. Exploiting the temporal correlation, we propose an attention-based bidirectional long short-term memory (BiLSTM) network to solve the MUD problem. The BiLSTM network creates a pattern of the device activation history using forward and reverse pass LSTMs, whereas the attention mechanism provides essential context to the device activation points. By doing so, a hierarchical pathway is followed for detecting active devices in a grant-free scenario. Then, by utilising the complex spreading sequences, blind data detection for the estimated active devices is performed. The proposed framework does not require prior knowledge of device sparsity levels and channels for performing MUD. The results show that the proposed network achieves better performance compared to existing benchmark schemes.

eess.SP↗

arXiv2019

Grant-free Non-orthogonal Multiple Access for IoT: A Survey

Massive machine-type communications (mMTC) is one of the main three focus areas in the 5th generation (5G) of mobile standards to enable connectivity of a massive number of internet of things (IoT) devices with little or no human intervention. In conventional human-type communications (HTC), due to the limited number of available radio resources and orthogonal/non-overlapping nature of existing resource allocation techniques, users need to compete for connectivity through a random access (RA) process, which may turn into a performance bottleneck in mMTC. In this context, non-orthogonal multiple access (NOMA) has emerged as a potential technology that allows overlapping of multiple users over a radio resource, thereby creating an opportunity to enable more autonomous and grant-free communication, where devices can transmit data whenever they need. The existing literature on NOMA schemes majorly considers centralized scheduling based HTC, where users are already connected, and various system parameters like spreading sequences, interleaving patterns, power control, etc., are predefined. Contrary to HTC, mMTC traffic is different with mostly uplink communication, small data size per device, diverse quality of service, autonomous nature, and massive number of devices. Hence, the signaling overhead and latency of centralized scheduling becomes a potential performance bottleneck. To tackle this, grant-free access is needed, where mMTC devices can autonomously transmit their data over randomly chosen radio resources. This article, in contrast to existing surveys, comprehensively discusses the recent advances in NOMA from a grant-free connectivity perspective. Moreover, related practical challenges and future directions are discussed.

eess.SP↗

arXiv2019

Protograph LDPC Code Design for Asynchronous Random Access

This work addresses the physical layer channel code design for an uncoordinated, frame- and slot-asynchronous random access protocol. Starting from the observation that collisions between two users yield very specific interference patterns, we define a surrogate channel model and propose different protograph low-density parity-check code designs. The proposed codes are both tested in a setup where the physical layer is abstracted, as well as on a more realistic channel model, where finite-length physical layer simulations of the entire asynchronous random access scheme, including decoding are carried out. We find that the abstracted physical layer model overestimates the performance when short blocks are considered. Additionally, the optimized codes show gains in supported channel traffic - a measure of the number of terminals that can be concurrently accommodated on the channel - of around 17% at a packet loss rate of 10^{-2} w.r.t. off-the-shelf codes.

cs.IT↗

arXiv2019

Structural Characteristics of Two-Sender Index Coding

This paper studies index coding with two senders. In this setup, source messages are distributed among the senders possibly with common messages. In addition, there are multiple receivers, with each receiver having some messages a priori, known as side-information, and requesting one unique message such that each message is requested by only one receiver. Index coding in this setup is called two-sender unicast index coding (TSUIC). The main goal is to find the shortest aggregate normalized codelength, which is expressed as the optimal broadcast rate. In this work, firstly, for a given TSUIC problem, we form three independent sub-problems each consisting of the only subset of the messages, based on whether the messages are available only in one of the senders or in both senders. Then we express the optimal broadcast rate of the TSUIC problem as a function of the optimal broadcast rates of those independent sub-problems. In this way, we discover the structural characteristics of TSUIC. For the proofs of our results, we utilize confusion graphs and coding techniques used in single-sender index coding. To adapt the confusion graph technique in TSUIC, we introduce a new graph-coloring approach that is different from the normal graph coloring, which we call two-sender graph coloring, and propose a way of grouping the vertices to analyze the number of colors used. We further determine a class of TSUIC instances where a certain type of side-information can be removed without affecting their optimal broadcast rates. Finally, we generalize the results of a class of TSUIC problems to multiple senders.

cs.IT↗

arXiv2018

Cooperative Multi-Sender Index Coding

In this paper, we propose a new coding scheme and establish new bounds on the capacity region for the multi-sender unicast index-coding problem. We revisit existing partitioned Distributed Composite Coding (DCC) proposed by Sadeghi et al. and identify its limitations in the implementation of multi-sender composite coding and in the strategy of sender partitioning. We then propose two new coding components to overcome these limitations and develop a multi-sender Cooperative Composite Coding (CCC). We show that CCC can strictly improve upon partitioned DCC, and is the key to achieve optimality for a number of index-coding instances. The usefulness of CCC and its special cases is illuminated via non-trivial examples, and the capacity region is established for each example. Comparisons between CCC and other non-cooperative schemes in recent works are also provided to further demonstrate the advantage of CCC.

cs.IT↗

arXiv2018

Centralized Caching with Unequal Cache Sizes

We address a centralized caching problem with unequal cache sizes. We consider a system with a server of files connected through a shared error-free link to a group of cache-enabled users where one subgroup has a larger cache size than the other. We propose an explicit caching scheme for the considered system aimed at minimizing the load of worst-case demands over the shared link. As suggested by numerical evaluations, our scheme improves upon the best existing explicit scheme by having a lower worst-case load; also, our scheme performs within a multiplicative factor of 1.11 from the scheme that can be obtained by solving an optimisation problem in which the number of parameters grows exponentially with the number of users.

cs.IT↗

arXiv2018

Interlinked Cycles for Index Coding: Generalizing Cycles and Cliques

We consider a graphical approach to index coding. While cycles have been shown to provide coding gain, only disjoint cycles and cliques (a specific type of overlapping cycles) have been exploited in existing literature. In this paper, we define a more general form of overlapping cycles, called the interlinked-cycle (IC) structure, that generalizes cycles and cliques. We propose a scheme, called the interlinked-cycle-cover (ICC) scheme, that leverages IC structures in digraphs to construct scalar linear index codes. We characterize a class of infinitely many digraphs where our proposed scheme is optimal over all linear and non-linear index codes. Consequently, for this class of digraphs, we indirectly prove that scalar linear index codes are optimal. Furthermore, we show that the ICC scheme can outperform all existing graph-based schemes (including partial-clique-cover and fractional-local-chromatic number schemes), and a random-coding scheme (namely, composite coding) for certain graphs.

cs.IT↗

arXiv2017

On the problem of non-zero word error rates for fixed-rate error correction codes in continuous variable quantum key distribution

The maximum operational range of continuous variable quantum key distribution protocols has shown to be improved by employing high-efficiency forward error correction codes. Typically, the secret key rate model for such protocols is modified to account for the non-zero word error rate of such codes. In this paper, we demonstrate that this model is incorrect: Firstly, we show by example that fixed-rate error correction codes, as currently defined, can exhibit efficiencies greater than unity. Secondly, we show that using this secret key model combined with greater than unity efficiency codes, implies that it is possible to achieve a positive secret key over an entanglement breaking channel - an impossible scenario. We then consider the secret key model from a post-selection perspective, and examine the implications for key rate if we constrain the forward error correction codes to operate at low word error rates.

quant-ph↗

arXiv2017

Raptor Codes for Higher-Order Modulation Using a Multi-Edge Framework

In this paper, we represent Raptor codes as multi-edge type low-density parity-check (MET-LDPC) codes, which gives a general framework to design them for higher-order modulation using MET density evolution. We then propose an efficient Raptor code design method for higher-order modulation, where we design distinct degree distributions for distinct bit levels. We consider a joint decoding scheme based on belief propagation for Raptor codes and also derive an exact expression for the stability condition. In several examples, we demonstrate that the higher-order modulated Raptor codes designed using the multi-edge framework outperform previously reported higher-order modulation codes in literature.

cs.IT↗

arXiv2017

Analysis and design of Raptor codes using a multi-edge framework

The focus of this paper is on the analysis and design of Raptor codes using a multi-edge framework. In this regard, we first represent the Raptor code as a multi-edge type low-density parity-check (METLDPC) code. This MET representation gives a general framework to analyze and design Raptor codes over a binary input additive white Gaussian noise channel using MET density evolution (MET-DE). We consider a joint decoding scheme based on the belief propagation (BP) decoding for Raptor codes in the multi-edge framework, and analyze the convergence behavior of the BP decoder using MET-DE. In joint decoding of Raptor codes, the component codes correspond to inner code and precode are decoded in parallel and provide information to each other. We also derive an exact expression for the stability of Raptor codes with joint decoding. We then propose an efficient Raptor code design method using the multi-edge framework, where we simultaneously optimize the inner code and the precode. Finally we consider performance-complexity trade-offs of Raptor codes using the multi-edge framework. Through density evolution analysis we show that the designed Raptor codes using the multi-edge framework outperform the existing Raptor codes in literature in terms of the realized rate.

cs.IT↗

arXiv2016

Graph-Theoretic Approaches to Two-Sender Index Coding

Consider a communication scenario over a noiseless channel where a sender is required to broadcast messages to multiple receivers, each having side information about some messages. In this scenario, the sender can leverage the receivers' side information during the encoding of messages in order to reduce the required transmissions. This type of encoding is called index coding. In this paper, we study index coding with two cooperative senders, each with some subset of messages, and multiple receivers, each requesting one unique message. The index coding in this setup is called two-sender unicast index coding (TSUIC). The main aim of TSUIC is to minimize the total number of transmissions required by the two senders. Based on graph-theoretic approaches, we prove that TSUIC is equivalent to single-sender unicast index coding (SSUIC) for some special cases. Moreover, we extend the existing schemes for SSUIC, viz., the cycle-cover scheme, the clique-cover scheme, and the local-chromatic scheme to the corresponding schemes for TSUIC.

cs.IT↗

arXiv2016

The DoF Region of the Three-Receiver MIMO Broadcast Channel with Side Information and Its Relation to Index Coding Capacity

We consider the three-receiver Gaussian multiple-input multiple-output (MIMO) broadcast channel with an arbitrary number of antennas at each of the transmitter and the receivers. We investigate the degrees-of-freedom (DoF) region of the channel when each receiver requests a private message, and may know some of the messages requested by the other receivers as receiver message side information (RMSI). We establish the DoF region of the channel for all 16 possible non-isomorphic RMSI configurations by deriving tight inner and outer bounds on the region. To derive the inner bounds, we first propose a scheme for each RMSI configuration which exploits both the null space and the side information of the receivers. We then use these schemes in conjunction with time sharing for 15 RMSI configurations, and with time sharing and two-symbol extension for the remaining one. To derive the outer bounds, we construct enhanced versions of the channel for each RMSI configuration, and upper bound their DoF region. After establishing the DoF region, in the case where all the nodes have the same number of antennas, we introduce some common properties of the DoF region, and the capacity region of the index coding problem.

cs.IT↗

arXiv2016

Optimization of Graph Based Codes for Belief Propagation Decoding

A low-density parity-check (LDPC) code is a linear block code described by a sparse parity-check matrix, which can be efficiently represented by a bipartite Tanner graph. The standard iterative decoding algorithm, known as belief propagation, passes messages along the edges of this Tanner graph. Density evolution is an efficient method to analyze the performance of the belief propagation decoding algorithm for a particular LDPC code ensemble, enabling the determination of a decoding threshold. The basic problem addressed in this work is how to optimize the Tanner graph so that the decoding threshold is as large as possible. We introduce a new code optimization technique which involves the search space range which can be thought of as minimizing randomness in differential evolution or limiting the search range in exhaustive search. This technique is applied to the design of good irregular LDPC codes and multiedge type LDPC codes.

cs.IT↗

arXiv2016

A Joint Optimization Technique for Multi-Edge Type LDPC Codes

This paper considers the optimization of multi-edge type low-density parity-check (METLDPC) codes to maximize the decoding threshold. We propose an algorithm to jointly optimize the node degree distribution and the multi-edge structure of MET-LDPC codes for given values of the maximum number of edge-types and maximum node degrees. This joint optimization is particularly important for MET-LDPC codes as it is not clear a priori which structures will be good. Using several examples, we demonstrate that the MET-LDPC codes designed by the proposed joint optimization algorithm exhibit improved decoding thresholds compared to previously reported MET-LDPC codes.

cs.IT↗

arXiv2016

New Density Evolution Approximation for LDPC and Multi-Edge Type LDPC Codes

This paper considers density evolution for lowdensity parity-check (LDPC) and multi-edge type low-density parity-check (MET-LDPC) codes over the binary input additive white Gaussian noise channel. We first analyze three singleparameter Gaussian approximations for density evolution and discuss their accuracy under several conditions, namely at low rates, with punctured and degree-one variable nodes. We observe that the assumption of symmetric Gaussian distribution for the density-evolution messages is not accurate in the early decoding iterations, particularly at low rates and with punctured variable nodes. Thus single-parameter Gaussian approximation methods produce very poor results in these cases. Based on these observations, we then introduce a new density evolution approximation algorithm for LDPC and MET-LDPC codes. Our method is a combination of full density evolution and a single-parameter Gaussian approximation, where we assume a symmetric Gaussian distribution only after density-evolution messages closely follow a symmetric Gaussian distribution. Our method significantly improves the accuracy of the code threshold estimation. Additionally, the proposed method significantly reduces the computational time of evaluating the code threshold compared to full density evolution thereby making it more suitable for code design.

cs.IT↗

arXiv2016

Massive Multiple Access Based on Superposition Raptor Codes for M2M Communications

Machine-to-machine (M2M) wireless systems aim to provide ubiquitous connectivity between machine type communication (MTC) devices without any human intervention. Given the exponential growth of MTC traffic, it is of utmost importance to ensure that future wireless standards are capable of handling this traffic. In this paper, we focus on the design of a very efficient massive access strategy for highly dense cellular networks with M2M communications. Several MTC devices are allowed to simultaneously transmit at the same resource block by incorporating Raptor codes and superposition modulation. This significantly reduces the access delay and improves the achievable system throughput. A simple yet efficient random access strategy is proposed to only detect the selected preambles and the number of devices which have chosen them. No device identification is needed in the random access phase which significantly reduces the signalling overhead. The proposed scheme is analyzed and the maximum number of MTC devices that can be supported in a resource block is characterized as a function of the message length, number of available resources, and the number of preambles. Simulation results show that the proposed scheme can effectively support a massive number of MTC devices for a limited number of available resources, when the message size is small.

cs.IT↗