arXiv subjects

Raymond W. Yeung

Publications and source records attributed to Raymond W. Yeung.

At least 19 recordsLinked to original sources

arXiv2026

Lossy Compression for Sparse Aggregation

We consider the problem of transmitting sparse local updates to the server in a distributed learning system. Specifically, the system consists of $n$ clients, each possessing a $k$-sparse $d$-dimensional local model, and a central server responsible for aggregating the clients' models into a global model. The goal is to characterize the tradeoff between the communication cost in the transmission from the clients to the server and the accuracy in aggregating the global model. We propose a compression scheme for sparse local models by concatenating a covering method and a sketching method. We also present a converse based on f-divergence, which strengthens the conventional Fano-type lower bounds. The proposed lower bound is tight for the frequency estimation case, that is, each coordinate takes values in a binary alphabet. For general alphabets, the proposed achievable schemes remain suboptimal relative to the converse bounds, indicating that a complete characterization of the communication-accuracy tradeoff requires further investigation.

cs.IT↗

arXiv2026

On Minimum Distances for Error Correction and Detection of Generalized Network Code

It is well known that the minimum distance for linear network codes plays the same role as the minimum distance for classical error control codes. However, Yang and Yeung (2008) discovered that for nonlinear network codes, the minimum distance for error correction is not always the same as the minimum distance for error detection. Inspired by the idea that the channel will affect the distances between the codewords, we establish the scheme of a generalized network channel and a generalized network code. Then, we systematically define the distances for error correction and error detection under the scheme of the generalized network code. We consider the joint error correction and detection in the generalized network code and obtain a complete characterization by introducing a distance and its refined version for this purpose. We enhance our understanding of the relation between various distances for error correction and detection in generalized network codes by proving some bounds on these distances.

cs.IT↗

arXiv2026

Characterizations of Conditional Mutual Independence: Equivalence and Implication

Conditional independence, and more generally conditional mutual independence, are central notions in probability theory. In their general forms, they include functional dependence as a special case. In this paper, we tackle two fundamental problems related to conditional mutual independence. Let $K$ and $K'$ be two conditional mutual independncies (CMIs) defined on a finite set of discrete random variables. We have obtained a necessary and sufficient condition for i) $K$ is equivalent to $K'$; ii) $K$ implies $K'$. These characterizations are in terms of a canonical form introduced for conditional mutual independence.

math.PR↗

arXiv2025

Secure Network Function Computation for Linear Functions, Part II: Target-Function Security

In this Part II of a two-part paper, we put forward secure network function computation, where in a directed acyclic network, a sink node is required to compute a target function of which the inputs are generated as source messages at multiple source nodes, while a wiretapper, who can access any one but not more than one wiretap set in a given collection of wiretap sets, is not allowed to obtain any information about a security function of the source messages. In Part I of the two-part paper, we have investigated securely computing linear functions with the wiretapper who can eavesdrop any edge subset up to a certain size r, referred to as the security level, where the security function is the identity function. The notion of this security is called source security. In the current paper, we consider another interesting model which is the same as the above one except that the security function is identical to the target function, i.e., we need to protect the information on the target function from being leaked to the wiretapper. The notion of this security is called target-function security. We first prove a non-trivial upper bound on the secure computing capacity, which is applicable to arbitrary network topologies and arbitrary security levels. In particular, when the security level r is equal to 0, the upper bound reduces to the computing capacity without security consideration. Further, from an algebraic point of view, we prove two equivalent conditions for target-function security and source security for the existence of the corresponding linear function-computing secure network codes. With them, for any linear function over a given finite field, we develop a code construction of linear secure network codes for target-function security and thus obtain a lower bound on the secure computing capacity; and also generalize the code construction developed in Part I for source security.

cs.IT↗

arXiv2025

Inequalities Revisited

In the past over two decades, very fruitful results have been obtained in information theory in the study of the Shannon entropy. This study has led to the discovery of a new class of constraints on the Shannon entropy called non-Shannon-type inequalities. Intimate connections between the Shannon entropy and different branches of mathematics including group theory, combinatorics, Kolmogorov complexity, probability, matrix theory, etc, have been established. All these discoveries were based on a formality introduced for constraints on the Shannon entropy, which suggested the possible existence of constraints that were not previously known. We assert that the same formality can be applied to inequalities beyond information theory. To illustrate the ideas, we revisit through the lens of this formality three fundamental inequalities in mathematics: the AM-GM inequality in algebra, Markov's inequality in probability theory, and the Cauchy-Scharwz inequality for inner product spaces. Applications of this formality have the potential of leading to the discovery of new inequalities and constraints in different branches of mathematics.

cs.IT↗

arXiv2025

Towards High-Performance Network Coding: FPGA Acceleration With Bounded-value Generators

Network coding enhances performance in network communications and distributed storage by increasing throughput and robustness while reducing latency. Batched Sparse (BATS) codes are a class of capacity-achieving network codes, but their practical applications are hindered by their structure, computational intensity, and power demands of finite field operations. Most literature focuses on algorithmic-level techniques to improve coding efficiency. Optimization with an algorithm/hardware co-designing approach has long been neglected. Leveraging the unique structure of BATS codes, we first present CS-BATS, a hardware-friendly variant. Next we propose a simple but effective bounded-value generator, to reduce the size of a finite field multiplier by up to 70%. Finally, we report on a scalable and resource-efficient FPGA-based network coding accelerator that achieves a throughput of 27 Gbps, a speedup of more than 300 over software.

cs.AR↗

arXiv2024

Ning Cai: A Tribute to a Pioneering Scholar in Information Theory

It is with heavy hearts that we mourn the passing of Ning Cai, a luminary whose pioneering spirit illuminated the realms of network coding and beyond. On May 25, 2023, at the age of 75, Prof. Cai bid farewell, leaving behind a profound legacy that continues to resonate across generations of researchers. His contributions spanned a vast spectrum, from the groundbreaking explorations in network coding to the intricate realms of quantum information theory. Ning's indelible mark on the academic landscape is a testament to his unwavering dedication and relentless pursuit of knowledge. Among his many accolades, Ning's seminal works garnered widespread recognition, exemplified by the prestigious 2005 IEEE Information Theory Society Paper Award for his work "Linear Network Coding." Furthermore, his enduring impact was underscored by the 2018 ACM SIGMOBILE Test-of-Time Paper Award, bestowed upon his paper "Network Information Flow." In addition to his scholarly achievements, Ning's unwavering commitment to mentorship has left an indelible mark on countless aspiring scholars. His guidance and wisdom continue to inspire and guide future generations in their scholarly pursuits. As we bid farewell to a titan in the field, let us cherish the legacy of Ning Cai, whose brilliance and generosity of spirit will forever endure in the annals of academia.

cs.IT↗

arXiv2024

Dependence Analysis and Structured Construction for Batched Sparse Code

In coding theory, codes are usually designed with a certain level of randomness to facilitate analysis and accommodate different channel conditions. However, the resulting random code constructed can be suboptimal in practical implementations. Represented by a bipartite graph, the Batched Sparse Code (BATS Code) is a randomly constructed erasure code that utilizes network coding to achieve near-optimal performance in wireless multi-hop networks. In the performance analysis in the previous research, it is implicitly assumed that the coded batches in the BATS code are independent. This assumption holds only asymptotically when the number of input symbols is infinite, but it does not generally hold in a practical setting where the number of input symbols is finite, especially when the code is constructed randomly. We show that dependence among the batches significantly degrades the code's performance. In order to control the batch dependence through graphical design, we propose constructing the BATS code in a structured manner. A hardware-friendly structured BATS code called the Cyclic-Shift BATS (CS-BATS) code is proposed, which constructs the code from a small base graph using light-weight cyclic-shift operations. We demonstrate that when the base graph is properly designed, a higher decoding rate and a smaller complexity can be achieved compared with the random BATS code.

cs.IT↗

arXiv2024

Proving Information Inequalities by Gaussian Elimination

The proof of information inequalities and identities under linear constraints on the information measures is an important problem in information theory. For this purpose, ITIP and other variant algorithms have been developed and implemented, which are all based on solving a linear program (LP). In this paper, we develop a method with symbolic computation. Compared with the known methods, our approach can completely avoids the use of linear programming which may cause numerical errors. Our procedures are also more efficient computationally.

cs.IT↗

arXiv2023

Reliable Throughput of Generalized Collision Channel without Synchronization

We consider a generalized collision channel model for general multi-user communication systems, an extension of Massey and Mathys' collision channel without feedback for multiple access communications. In our model, there are multiple transmitters and receivers sharing the same communication channel. The transmitters are not synchronized and arbitrary time offsets between transmitters and receivers are assumed. A ``collision" occurs if two or more packets from different transmitters partially or completely overlap at a receiver. Our model includes the original collision channel as a special case. This paper focuses on reliable throughputs that are approachable for arbitrary time offsets. We consider both slot-synchronized and non-synchronized cases and characterize their reliable throughput regions for the generalized collision channel model. These two regions are proven to coincide. Moreover, it is shown that the protocol sequences constructed for multiple access communication remain ``throughput optimal" in the generalized collision channel model. We also identify the protocol sequences that can approach the outer boundary of the reliable throughput region.

cs.IT↗

arXiv2022

Secure Network Function Computation for Linear Functions -- Part I: Source Security

In this paper, we put forward secure network function computation over a directed acyclic network. In such a network, a sink node is required to compute with zero error a target function of which the inputs are generated as source messages at multiple source nodes, while a wiretapper, who can access any one but not more than one wiretap set in a given collection of wiretap sets, is not allowed to obtain any information about a security function of the source messages. The secure computing capacity for the above model is defined as the maximum average number of times that the target function can be securely computed with zero error at the sink node with the given collection of wiretap sets and security function for one use of the network. The characterization of this capacity is in general overwhelmingly difficult. In the current paper, we consider securely computing linear functions with a wiretapper who can eavesdrop any subset of edges up to a certain size r, referred to as the security level, with the security function being the identity function. We first prove an upper bound on the secure computing capacity, which is applicable to arbitrary network topologies and arbitrary security levels. When the security level r is equal to 0, our upper bound reduces to the computing capacity without security consideration. We discover the surprising fact that for some models, there is no penalty on the secure computing capacity compared with the computing capacity without security consideration. We further obtain an equivalent expression of the upper bound by using a graph-theoretic approach, and accordingly we develop an efficient approach for computing this bound. Furthermore, we present a construction of linear function-computing secure network codes and obtain a lower bound on the secure computing capacity.

cs.IT↗

arXiv2022

Proving Information Inequalities and Identities with Symbolic Computation

Proving linear inequalities and identities of Shannon's information measures, possibly with linear constraints on the information measures, is an important problem in information theory. For this purpose, ITIP and other variant algorithms have been developed and implemented, which are all based on solving a linear program (LP). In particular, an identity $f = 0$ is verified by solving two LPs, one for $f \ge 0$ and one for $f \le 0$. In this paper, we develop a set of algorithms that can be implemented by symbolic computation. Based on these algorithms, procedures for verifying linear information inequalities and identities are devised. Compared with LP-based algorithms, our procedures can produce analytical proofs that are both human-verifiable and free of numerical errors. Our procedures are also more efficient computationally. For constrained inequalities, by taking advantage of the algebraic structure of the problem, the size of the LP that needs to be solved can be significantly reduced. For identities, instead of solving two LPs, the identity can be verified directly with very little computation.

cs.IT↗

arXiv2021

Intrablock Interleaving for Batched Network Coding with Blockwise Adaptive Recoding

Batched network coding (BNC) is a low-complexity solution to network transmission in multi-hop packet networks with packet loss. BNC encodes the source data into batches of packets. As a network coding scheme, the intermediate nodes perform recoding on the received packets belonging to the same batch instead of just forwarding them. A recoding scheme that may generate more recoded packets for batches of a higher rank is also called adaptive recoding. Meanwhile, in order to combat burst packet loss, the transmission of a block of batches can be interleaved. Stream interleaving studied in literature achieves the maximum separation among any two consecutive packets of a batch, but permutes packets across blocks and hence cannot bound the buffer size and the latency. To resolve the issue of stream interleaver, we design an intrablock interleaver for adaptive recoding that can preserve the advantages of using a block interleaver when the number of recoded packets is the same for all batches. We use potential energy in classical mechanics to measure the performance of an interleaver, and propose an algorithm to optimize the interleaver with this performance measure. Our problem formulation and algorithm for intrablock interleaving are also of independent interest.

cs.IT↗

arXiv2021

Linear Network Error Correction Coding: A Revisit

We consider linear network error correction (LNEC) coding when errors may occur on edges of a communication network of which the topology is known. In this paper, we first revisit and explore the framework of LNEC coding, and then unify two well-known LNEC coding approaches. Furthermore, by developing a graph-theoretic approach to the framework of LNEC coding, we obtain a significantly enhanced characterization of the error correction capability of LNEC codes in terms of the minimum distances at the sink nodes. In LNEC coding, the minimum required field size for the existence of LNEC codes, in particular LNEC maximum distance separable (MDS) codes which are a type of most important optimal codes, is an open problem not only of theoretical interest but also of practical importance, because it is closely related to the implementation of the coding scheme in terms of computational complexity and storage requirement. By applying the graph-theoretic approach, we obtain an improved upper bound on the minimum required field size. The improvement over the existing results is in general significant. The improved upper bound, which is graph-theoretic, depends only on the network topology and requirement of the error correction capability but not on a specific code construction. However, this bound is not given in an explicit form. We thus develop an efficient algorithm that can compute the bound in linear time. In developing the upper bound and the efficient algorithm for computing this bound, various graph-theoretic concepts are introduced. These concepts appear to be of fundamental interest in graph theory and they may have further applications in graph theory and beyond.

cs.IT↗

arXiv2020

Weakly Secure Symmetric Multilevel Diversity Coding

Multilevel diversity coding is a classical coding model where multiple mutually independent information messages are encoded, such that different reliability requirements can be afforded to different messages. It is well known that {\em superposition coding}, namely separately encoding the independent messages, is optimal for symmetric multilevel diversity coding (SMDC) (Yeung-Zhang 1999). In the current paper, we consider weakly secure SMDC where security constraints are injected on each individual message, and provide a complete characterization of the conditions under which superposition coding is sum-rate optimal. Two joint coding strategies, which lead to rate savings compared to superposition coding, are proposed, where some coding components for one message can be used as the encryption key for another. By applying different variants of Han's inequality, we show that the lack of opportunity to apply these two coding strategies directly implies the optimality of superposition coding. It is further shown that under a set of particular security constraints, one of the proposed joint coding strategies can be used to construct a code that achieves the optimal rate region.

cs.IT↗

arXiv2019

The Explicit Coding Rate Region of Symmetric Multilevel Diversity Coding

It is well known that {\em superposition coding}, namely separately encoding the independent sources, is optimal for symmetric multilevel diversity coding (SMDC) (Yeung-Zhang 1999). However, the characterization of the coding rate region therein involves uncountably many linear inequalities and the constant term (i.e., the lower bound) in each inequality is given in terms of the solution of a linear optimization problem. Thus this implicit characterization of the coding rate region does not enable the determination of the achievability of a given rate tuple. In this paper, we first obtain closed-form expressions of these uncountably many inequalities. Then we identify a finite subset of inequalities that is sufficient for characterizing the coding rate region. This gives an explicit characterization of the coding rate region. We further show by the symmetry of the problem that only a much smaller subset of this finite set of inequalities needs to be verified in determining the achievability of a given rate tuple. Yet, the cardinality of this smaller set grows at least exponentially fast with $L$. We also present a subset entropy inequality, which together with our explicit characterization of the coding rate region, is sufficient for proving the optimality of superposition coding.

cs.IT↗

arXiv2018

Local-Encoding-Preserving Secure Network Coding---Part I: Fixed Security Level

Information-theoretic security is considered in the paradigm of network coding in the presence of wiretappers, who can access one arbitrary edge subset up to a certain size, also referred to as the security level. Secure network coding is applied to prevent the leakage of the source information to the wiretappers. In this two-part paper, we consider the problem of secure network coding when the information rate and the security level can change over time. In the current paper (i.e., Part I of the two-part paper), we focus on the problem for a fixed security level and a flexible rate. To efficiently solve this problem, we put forward local-encoding-preserving secure network coding, where a family of secure linear network codes (SLNCs) is called local-encoding-preserving if all the SLNCs in this family share a common local encoding kernel at each intermediate node in the network. We present an efficient approach for constructing upon an SLNC that exists a local-encoding-preserving SLNC with the same security level and the rate reduced by one. By applying this approach repeatedly, we can obtain a family of local-encoding-preserving SLNCs with a fixed security level and multiple rates. We also develop a polynomial-time algorithm for efficient implementation of this approach. Furthermore, it is proved that the proposed approach incurs no penalty on the required field size for the existence of SLNCs in terms of the best known lower bound by Guang and Yeung. The result in this paper will be used as a building block for efficiently constructing a family of local-encoding-preserving SLNCs for all possible pairs of rate and security level, which will be discussed in the companion paper (i.e., Part II of the two-part paper).

cs.IT↗

arXiv2018

Local-Encoding-Preserving Secure Network Coding---Part II: Flexible Rate and Security Level

In the two-part paper, we consider the problem of secure network coding when the information rate and the security level can change over time. To efficiently solve this problem, we put forward local-encoding-preserving secure network coding, where a family of secure linear network codes (SLNCs) is called local-encoding-preserving (LEP) if all the SLNCs in this family share a common local encoding kernel at each intermediate node in the network. In this paper (Part II), we first consider the design of a family of LEP SLNCs for a fixed rate and a flexible security level. We present a novel and efficient approach for constructing upon an SLNC that exists an LEP SLNC with the same rate and the security level increased by one. Next, we consider the design of a family of LEP SLNCs for a fixed dimension (equal to the sum of rate and security level) and a flexible pair of rate and security level. We propose another novel approach for designing an SLNC such that the same SLNC can be applied for all the rate and security-level pairs with the fixed dimension. Also, two polynomial-time algorithms are developed for efficient implementations of our two approaches, respectively. Furthermore, we prove that both approaches do not incur any penalty on the required field size for the existence of SLNCs in terms of the best known lower bound by Guang and Yeung. Finally, we consider the ultimate problem of designing a family of LEP SLNCs that can be applied to all possible pairs of rate and security level. By combining the construction of a family of LEP SLNCs for a fixed security level and a flexible rate (obtained in Part I) with the constructions of the two families of LEP SLNCs in the current paper in suitable ways, we can obtain a family of LEP SLNCs that can be applied for all possible pairs of rate and security level. Three possible such constructions are presented.

cs.IT↗