arXiv subjects

Muriel Medard

Publications and source records attributed to Muriel Medard.

At least 19 recordsLinked to original sources

arXiv2025

Decoding in the presence of ISI without interleaving -- ORBGRAND-AI

Inter symbol interference (ISI), which occurs in a wide variety of channels, is a result of time dispersion. It can be mitigated by equalization, which results in noise coloring. Inspired by the development of Approximate Independence in statistical physics, for such colored noise we propose a decoder called Ordered Reliability Bits Guessing Random Additive Noise Decoding (ORBGRAND-AI) that operates without the need for turbo equalization or interleaving. By foregoing interleaving, ORBGRAND-AI can deliver the same, or lower, block error rate (BLER) for the same amount of energy per information bit in an ISI channel as a state-of-the-art soft input decoder, such as Cyclic Redundancy Check Assisted-Successive Cancellation List (CA-SCL) decoding, with an interleaver. To assess the decoding performance of ORBGRAND-AI, we consider delay tap models and their associated colored noise. In particular, we examine a two-tap dicode ISI channel as well as an ISI channel derived from data from RFView, a physics-informed modeling and simulation tool. We investigate the dicode and RFView channel under a variety of imperfect channel state information assumptions and show that a second order autoregressive model adequately represents the RFView channel effect.

eess.SP↗

arXiv2025

From Indexing to Coding: A New Paradigm for Data Availability Sampling

The data availability problem is a central challenge in blockchain systems and lies at the core of the accessibility and scalability issues faced by platforms such as Ethereum. Modern solutions employ several approaches, with data availability sampling (DAS) being the most self-sufficient and minimalistic in its security assumptions. Existing DAS methods typically form cryptographic commitments on codewords of fixed-rate erasure codes, which restrict light nodes to sampling from a predetermined set of coded symbols. In this paper, we introduce a new approach to DAS that modularizes the coding and commitment process by committing to the uncoded data while performing sampling through on-the-fly coding. The resulting samples are significantly more expressive, enabling light nodes to obtain, in concrete implementations, up to multiple orders of magnitude stronger assurances of data availability than from sampling pre-committed symbols from a fixed-rate redundancy code as done in established DAS schemes using Reed Solomon or low density parity check codes. We present a concrete protocol that realizes this paradigm using random linear network coding (RLNC).

cs.CR↗

arXiv2025

OPTIMUMP2P: Fast and Reliable Gossiping in P2P Networks

Gossip algorithms are pivotal in the dissemination of information within decentralized systems. Consequently, numerous gossip libraries have been developed and widely utilized especially in blockchain protocols for the propagation of blocks and transactions. A well-established library is libp2p, which provides two gossip algorithms: floodsub and gossipsub. These algorithms enable the delivery of published messages to a set of peers. In this work we aim to enhance the performance and reliability of libp2p by introducing OPTIMUMP2P, a novel gossip algorithm that leverages the capabilities of Random Linear Network Coding (RLNC) to expedite the dissemination of information in a peer-to-peer (P2P) network while ensuring reliable delivery, even in the presence of malicious actors capable of corrupting the transmitted data. Preliminary research from the Ethereum Foundation has demonstrated the use of RLNC in the significant improvement in the block propagation time [14]. Here we present extensive evaluation results both in simulation and real-world environments that demonstrate the performance gains of OPTIMUMP2P over the Gossipsub protocol.

cs.DC↗

arXiv2024

Using a Single-Parity-Check to Reduce the Guesswork of Guessing Codeword Decoding

Guessing Codeword Decoding (GCD) is a recently proposed soft-input forward error correction decoder for arbitrary binary linear codes. Inspired by recent proposals that leverage binary linear codebook structure to reduce the number of queries made by Guessing Random Additive Noise Decoding (GRAND), for binary linear codes that include a full-message single parity-check (SPC) bit, we show that it is possible to reduce the number of queries made by GCD by a factor of up to 2 with the greatest guesswork reduction realized at lower SNRs, without impacting decoding precision. Codes without a full-message SPC can be modified to include one by changing a column of the generator matrix to obtain a decoding complexity advantage, and we demonstrate that this can often be done without losing decoding precision. To practically avail of the complexity advantage, a noise effect pattern generator capable of producing sequences for given Hamming weights, such as the landslide algorithm developed for ORBGRAND, is necessary.

cs.IT↗

arXiv2024

Turbo product decoding of cubic tensor codes

Long, powerful soft detection forward error correction codes are typically constructed by concatenation of shorter component codes that are decoded through iterative Soft-Input Soft-Output (SISO) procedures. The current gold-standard is Low Density Parity Check (LDPC) codes, which are built from weak single parity check component codes that are capable of producing accurate SO. Due to the recent development of SISO decoders that produce highly accurate SO with codes that have multiple redundant bits, square product code constructions that can avail of more powerful component codes have been shown to be competitive with the LDPC codes in the 5G New Radio standard in terms of decoding performance while requiring fewer iterations to converge. Motivated by applications that require more powerful low-rate codes, in the present paper we explore the possibility of extending this design space by considering the construction and decoding of cubic tensor codes.

cs.IT↗

arXiv2024

Soft-output (SO) GRAND and Iterative Decoding to Outperform LDPCs

We establish that a large, flexible class of long, high redundancy error correcting codes can be efficiently and accurately decoded with guessing random additive noise decoding (GRAND). Performance evaluation demonstrates that it is possible to construct simple product codes with lengths of approximately 200 to 4000 bits and rates between 0.2 and 0.8 that outperform low-density parity-check (LDPC) codes from the 5G New Radio standard in both AWGN and fading channels. The concatenated structure enables many desirable features, including: low-complexity hardware-friendly encoding and decoding; significant flexibility in length and rate through modularity; and high levels of parallelism in encoding and decoding that enable low latency. Central is the development of a method through which any soft-input (SI) GRAND algorithm can provide soft-output (SO) in the form of an accurate a-posteriori estimate of the likelihood that a decoding is correct or, in the case of list decoding, the likelihood that each element of the list is correct. The distinguishing feature of soft-output GRAND (SOGRAND) is the provision of an estimate that the correct decoding has not been found, even when providing a single decoding. That per-block SO can be converted into accurate per-bit SO by a weighted sum that includes a term for the SI. Implementing SOGRAND adds negligible computation and memory to the existing decoding process, and using it results in a practical, low-latency alternative to LDPC codes.

cs.IT↗

arXiv2024

MultiTok: Variable-Length Tokenization for Efficient LLMs Adapted from LZW Compression

Large language models have drastically changed the prospects of AI by introducing technologies for more complex natural language processing. However, current methodologies to train such LLMs require extensive resources including but not limited to large amounts of data, expensive machinery, and lengthy training. To solve this problem, this paper proposes a new tokenization method inspired by universal Lempel-Ziv-Welch data compression that compresses repetitive phrases into multi-word tokens. With MultiTok as a new tokenizing tool, we show that language models are able to be trained notably more efficiently while offering a similar accuracy on more succinct and compressed training data. In fact, our results demonstrate that MultiTok achieves a comparable performance to the BERT and GPT standards as both a stand-alone tokenizer and an add-on to existing tokenizers while also providing close to 2.5x faster training with more than 30% less training data.

cs.CL↗

arXiv2024

Soft-output Guessing Codeword Decoding

We establish that it is possible to extract accurate blockwise and bitwise soft output from Guessing Codeword Decoding with minimal additional computational complexity by considering it as a variant of Guessing Random Additive Noise Decoding. Blockwise soft output can be used to control decoding misdetection rate while bitwise soft output results in a soft-input soft-output decoder that can be used for efficient iterative decoding of long, high redundancy codes.

cs.IT↗

arXiv2024

TexShape: Information Theoretic Sentence Embedding for Language Models

With the exponential growth in data volume and the emergence of data-intensive applications, particularly in the field of machine learning, concerns related to resource utilization, privacy, and fairness have become paramount. This paper focuses on the textual domain of data and addresses challenges regarding encoding sentences to their optimized representations through the lens of information-theory. In particular, we use empirical estimates of mutual information, using the Donsker-Varadhan definition of Kullback-Leibler divergence. Our approach leverages this estimation to train an information-theoretic sentence embedding, called TexShape, for (task-based) data compression or for filtering out sensitive information, enhancing privacy and fairness. In this study, we employ a benchmark language model for initial text representation, complemented by neural networks for information-theoretic compression and mutual information estimations. Our experiments demonstrate significant advancements in preserving maximal targeted information and minimal sensitive information over adverse compression ratios, in terms of predictive accuracy of downstream models that are trained using the compressed data.

cs.CL↗

arXiv2024

Successive Refinement in Large-Scale Computation: Advancing Model Inference Applications

Modern computationally-intensive applications often operate under time constraints, necessitating acceleration methods and distribution of computational workloads across multiple entities. However, the outcome is either achieved within the desired timeline or not, and in the latter case, valuable resources are wasted. In this paper, we introduce solutions for layered-resolution computation. These solutions allow lower-resolution results to be obtained at an earlier stage than the final result. This innovation notably enhances the deadline-based systems, as if a computational job is terminated due to time constraints, an approximate version of the final result can still be generated. Moreover, in certain operational regimes, a high-resolution result might be unnecessary, because the low-resolution result may already deviate significantly from the decision threshold, for example in AI-based decision-making systems. Therefore, operators can decide whether higher resolution is needed or not based on intermediate results, enabling computations with adaptive resolution. We present our framework for two critical and computationally demanding jobs: distributed matrix multiplication (linear) and model inference in machine learning (nonlinear). Our theoretical and empirical results demonstrate that the execution delay for the first resolution is significantly shorter than that for the final resolution, while maintaining overall complexity comparable to the conventional one-shot approach. Our experiments further illustrate how the layering feature increases the likelihood of meeting deadlines and enables adaptability and transparency in massive, large-scale computations.

cs.IT↗

arXiv2023

InfoShape: Task-Based Neural Data Shaping via Mutual Information

The use of mutual information as a tool in private data sharing has remained an open challenge due to the difficulty of its estimation in practice. In this paper, we propose InfoShape, a task-based encoder that aims to remove unnecessary sensitive information from training data while maintaining enough relevant information for a particular ML training task. We achieve this goal by utilizing mutual information estimators that are based on neural networks, in order to measure two performance metrics, privacy and utility. Using these together in a Lagrangian optimization, we train a separate neural network as a lossy encoder. We empirically show that InfoShape is capable of shaping the encoded samples to be informative for a specific downstream task while eliminating unnecessary sensitive information. Moreover, we demonstrate that the classification accuracy of downstream models has a meaningful connection with our utility and privacy measures.

cs.IT↗

arXiv2023

Soft detection physical layer insecurity

We establish that during the execution of any Guessing Random Additive Noise Decoding (GRAND) algorithm, an interpretable, useful measure of decoding confidence can be evaluated. This measure takes the form of a log-likelihood ratio (LLR) of the hypotheses that, should a decoding be found by a given query, the decoding is correct versus its being incorrect. That LLR can be used as soft output for a range of applications and we demonstrate its utility by showing that it can be used to confidently discard likely erroneous decodings in favor of returning more readily managed erasures. We show that feature can be used to compromise the physical layer security of short length wiretap codes by accurately and confidently revealing a proportion of a communication when code-rate is far above the Shannon capacity of the associated hard detection channel.

cs.IT↗

arXiv2023

Using channel correlation to improve decoding -- ORBGRAND-AI

To meet the Ultra Reliable Low Latency Communication (URLLC) needs of modern applications, there have been significant advances in the development of short error correction codes and corresponding soft detection decoders. A substantial hindrance to delivering low-latency is, however, the reliance on interleaving to break up omnipresent channel correlations to ensure that decoder input matches decoder assumptions. Consequently, even when using short codes, the need to wait to interleave data at the sender and de-interleave at the receiver results in significant latency that acts contrary to the goals of URLLC. Moreover, interleaving reduces channel capacity, so that potential decoding performance is degraded. Here we introduce a variant of Ordered Reliability Bits Guessing Random Additive Noise Decoding (ORBGRAND), which we call ORBGRAND-Approximate Independence (ORBGRAND-AI), a soft-detection decoder that can decode any moderate redundancy code and overcomes the limitation of existing decoding paradigms by leveraging channel correlations and circumventing the need for interleaving. By leveraging correlation, not only is latency reduced, but error correction performance can be enhanced by multiple dB, while decoding complexity is also reduced, offering one potential solution for the provision of URLLC.

cs.IT↗

arXiv2023

GRAND-EDGE: A Universal, Jamming-resilient Algorithm with Error-and-Erasure Decoding

Random jammers that overpower transmitted signals are a practical concern for many wireless communication protocols. As such, wireless receivers must be able to cope with standard channel noise and jamming (intentional or unintentional). To address this challenge, we propose a novel method to augment the resilience of the recent family of universal error-correcting GRAND algorithms. This method, called Erasure Decoding by Gaussian Elimination (EDGE), impacts the syndrome check block and is applicable to any variant of GRAND. We show that the proposed EDGE method naturally reverts to the original syndrome check function in the absence of erasures caused by jamming. We demonstrate this by implementing and evaluating GRAND-EDGE and ORBGRAND-EDGE. Simulation results, using a Random Linear Code (RLC) with a code rate of $105/128$, show that the EDGE variants lower both the Block Error Rate (BLER) and the computational complexity by up to five order of magnitude compared to the original GRAND and ORBGRAND algorithms. We further compare ORBGRAND-EDGE to Ordered Statistics Decoding (OSD), and demonstrate an improvement of up to three orders of magnitude in the BLER.

cs.IT↗

arXiv2022

A General Security Approach for Soft-information Decoding against Smart Bursty Jammers

Malicious attacks such as jamming can cause significant disruption or complete denial of service (DoS) to wireless communication protocols. Moreover, jamming devices are getting smarter, making them difficult to detect. Forward error correction, which adds redundancy to data, is commonly deployed to protect communications against the deleterious effects of channel noise. Soft-information error correction decoders obtain reliability information from the receiver to inform their decoding, but in the presence of a jammer such information is misleading and results in degraded error correction performance. As decoders assume noise occurs independently to each bit, a bursty jammer will lead to greater degradation in performance than a non-bursty one. Here we establish, however, that such temporal dependencies can aid inferences on which bits have been subjected to jamming, thus enabling counter-measures. In particular, we introduce a pre-decoding processing step that updates log-likelihood ratio (LLR) reliability information to reflect inferences in the presence of a jammer, enabling improved decoding performance for any soft detection decoder. The proposed method requires no alteration to the decoding algorithm. Simulation results show that the method correctly infers a significant proportion of jamming in any received frame. Results with one particular decoding algorithm, the recently introduced ORBGRAND, show that the proposed method reduces the block-error rate (BLER) by an order of magnitude for a selection of codes, and prevents complete DoS at the receiver.

cs.IT↗

arXiv2022

Ordered Reliability Bits Guessing Random Additive Noise Decoding

Error correction techniques traditionally focus on the co-design of restricted code-structures in tandem with code-specific decoders that are computationally efficient when decoding long codes in hardware. Modern applications are, however, driving demand for ultra-reliable low-latency communications (URLLC), rekindling interest in the performance of shorter, higher-rate error correcting codes, and raising the possibility of revisiting universal, code-agnostic decoders. To that end, here we introduce a soft-detection variant of Guessing Random Additive Noise Decoding (GRAND) called Ordered Reliability Bits GRAND that can accurately decode any moderate redundancy block-code. It is designed with efficient circuit implementation in mind, and determines accurate decodings while retaining the original hard detection GRAND algorithm's suitability for a highly parallelized implementation in hardware. ORBGRAND is shown to provide excellent soft decision block error performance for codes of distinct classes (BCH, CA-Polar and RLC) with modest complexity, while providing better block error rate performance than CA-SCL, a state of the art soft detection CA-Polar decoder. ORBGRAND offers the possibility of an accurate, energy efficient soft detection decoder suitable for delivering URLLC in a single hardware realization.

cs.IT↗

arXiv2022

Soft decoding without soft demapping with ORBGRAND

For spectral efficiency, higher order modulation symbols confer information on more than one bit. As soft detection forward error correction decoders assume the availability of information at binary granularity, however, soft demappers are required to compute per-bit reliabilities from complex-valued signals. Here we show that the recently introduced universal soft detection decoder ORBGRAND can be adapted to work with symbol-level soft information, obviating the need for energy expensive soft demapping. We establish that doing so reduces complexity while retaining the error correction performance achieved with the optimal demapper.

cs.IT↗

arXiv2022

Wideband Time Frequency Coding

In the wideband regime, the performance of many of the popular modulation schemes such as code division multiple access and orthogonal frequency division multiplexing falls quickly without channel state information. Obtaining the amount of channel information required for these techniques to work is costly and difficult, which suggests the need for schemes which can perform well without channel state information. In this work, we present one such scheme, called wideband time frequency coding, which achieves rates on the order of the additive white Gaussian noise capacity without requiring any channel state information. Wideband time frequency coding combines impulsive frequency shift keying with pulse position modulation, which allows for information to be encoded in both the transmitted frequency and the transmission time period. On the detection side, we propose a non-coherent decoder based on a square-law detector, akin to the optimal decoder for frequency shift keying based signals. The impacts of various parameters on the symbol error probability and capacity of wideband time frequency coding are investigated, and the results show that it is robust to shadowing and highly fading channels. When compared to other modulation schemes such as code division multiple access, orthogonal frequency division multiplexing, pulse position modulation, and impulsive frequency shift keying without channel state information, wideband time frequency coding achieves higher rates in the wideband regime, and performs comparably in smaller bandwidths.

cs.IT↗