arXiv subjects

Chung Chan

Publications and source records attributed to Chung Chan.

At least 19 recordsLinked to original sources

arXiv2021

Wiretap Secret Key Agreement Via Secure Omniscience

In this paper, we explore the connection between secret key agreement and secure omniscience within the setting of the multiterminal source model with a wiretapper who has side information. While the secret key agreement problem considers the generation of a maximum-rate secret key through public discussion, the secure omniscience problem is concerned with communication protocols for omniscience that minimize the rate of information leakage to the wiretapper. The starting point of our work is a lower bound on the minimum leakage rate for omniscience, $R_{\mathop{\mathrm{L}}}$, in terms of the wiretap secret key capacity, $C_{\mathop{\mathrm{W}}}$. Our interest is in identifying broad classes of sources for which this lower bound is met with equality, in which case we say that there is a duality between secure omniscience and secret key agreement. We show that this duality holds in the case of certain finite linear source (FLS) models, such as two-terminal FLS models and pairwise independent network models on trees with a linear wiretapper. Duality also holds for any FLS model in which $C_{\mathop{\mathrm{W}}}$ is achieved by a perfect linear secret key agreement scheme. We conjecture that the duality in fact holds unconditionally for any FLS model. On the negative side, we give an example of a (non-FLS) source model for which duality does not hold if we limit ourselves to communication-for-omniscience protocols with at most two (interactive) communications. We also address the secure function computation problem and explore the connection between the minimum leakage rate for computing a function and the wiretap secret key capacity.

cs.IT↗

arXiv2021

Targeted Gradient Descent: A Novel Method for Convolutional Neural Networks Fine-tuning and Online-learning

A convolutional neural network (ConvNet) is usually trained and then tested using images drawn from the same distribution. To generalize a ConvNet to various tasks often requires a complete training dataset that consists of images drawn from different tasks. In most scenarios, it is nearly impossible to collect every possible representative dataset as a priori. The new data may only become available after the ConvNet is deployed in clinical practice. ConvNet, however, may generate artifacts on out-of-distribution testing samples. In this study, we present Targeted Gradient Descent (TGD), a novel fine-tuning method that can extend a pre-trained network to a new task without revisiting data from the previous task while preserving the knowledge acquired from previous training. To a further extent, the proposed method also enables online learning of patient-specific data. The method is built on the idea of reusing a pre-trained ConvNet's redundant kernels to learn new knowledge. We compare the performance of TGD to several commonly used training approaches on the task of Positron emission tomography (PET) image denoising. Results from clinical images show that TGD generated results on par with training-from-scratch while significantly reducing data preparation and network training time. More importantly, it enables online learning on the testing study to enhance the network's generalization capability in real-world applications.

cs.CV↗

arXiv2021

Secret Key Agreement and Secure Omniscience of Tree-PIN Source with Linear Wiretapper

While the wiretap secret key capacity remains unknown for general source models even in the two-user case, we obtained a single-letter characterization for a large class of multi-user source models with a linear wiretapper who can observe any linear combinations of the source. We introduced the idea of irreducible sources to show existence of an optimal communication scheme that achieves perfect omniscience with minimum leakage of information to the wiretapper. This implies a duality between the problems of wiretap secret key agreement and secure omniscience, and such duality potentially holds for more general sources.

cs.IT↗

arXiv2020

Determining Optimal Rates for Communication for Omniscience

This paper considers the communication for omniscience (CO) problem: A set of users observe a discrete memoryless multiple source and want to recover the entire multiple source via noise-free broadcast communications. We study the problem of how to determine an optimal rate vector that attains omniscience with the minimum sum-rate, the total number of communications. The results cover both asymptotic and non-asymptotic models where the transmission rates are real and integral, respectively. We propose a modified decomposition algorithm (MDA) and a sum-rate increment algorithm (SIA) for the asymptotic and non-asymptotic models, respectively, both of which determine the value of the minimum sum-rate and a corresponding optimal rate vector in polynomial time. For the coordinate saturation capacity (CoordSatCap) algorithm, a nesting algorithm in MDA and SIA, we propose to implement it by a fusion method and show by experimental results that this fusion method contributes to a reduction in computation complexity. Finally, we show that the separable convex minimization problem over the optimal rate vector set in the asymptotic model can be decomposed by the fundamental partition, the optimal partition of the user set that determines the minimum sum-rate, so that the problem can be solved more efficiently.

cs.IT↗

arXiv2019

Secret key agreement for hypergraphical sources with limited total discussion

This work considers the problem of multiterminal secret key agreement by limited total public discussion under the hypergraphical source model. The secrecy capacity as a function of the total discussion rate is completely characterized by a polynomial-time computable linear program. Compared to the existing solution for a particular hypergraphical source model called the pairwise independent network (PIN) model, the current result is a non-trivial extension as it applies to a strictly larger class of sources and a more general scenario involving helpers and wiretapper's side information. In particular, while the existing solution by tree-packing can be strictly suboptimal for the PIN model with helpers and the hypergraphical source model in general, we can show that decremental secret key agreement and linear network coding is optimal, resolving a previous conjecture in the affirmative. The converse is established by a single-letter upper bound on the secrecy capacity for discrete memoryless multiple sources and individual discussion rate constraints. The minimax optimization involved in the bound can be relaxed to give the best existing upper bounds on secrecy capacities such as the lamination bounds for hypergraphical sources, helper-set bound for general sources, the bound at asymptotically zero discussion rate via the multivariate Gác--Körner common information, and the lower bound on communication complexity via a multivariate extension of the Wyner common information. These reductions unify existing bounding techniques and reveal surprising connections between seemingly different information-theoretic notions. Further challenges are posed in this work along with a simple example of finite linear source where the current converse techniques fail even though the proposed achieving scheme remains optimal.

cs.IT↗

arXiv2019

Secret Key Generation for Minimally Connected Hypergraphical Sources

This paper investigates the secret key generation in the multiterminal source model, where users observing correlated sources discuss interactively under limited rates to agree on a secret key. We focus on a class of sources representable by minimally connected hypergraphs. For such sources, we give a single-letter explicit characterization of the region of achievable secret key rate and public discussion rate tuple. This is the first result that completely characterizes the achievable rate region for a multiterminal source model, which is beyond the PIN model on a tree. We also obtain an explicit formula for the maximum achievable secret key rate, called the constrained secrecy capacity, as a function of the total discussion rate.

cs.IT↗

arXiv2019

Neural Entropic Estimation: A faster path to mutual information estimation

We point out a limitation of the mutual information neural estimation (MINE) where the network fails to learn at the initial training phase, leading to slow convergence in the number of training iterations. To solve this problem, we propose a faster method called the mutual information neural entropic estimation (MI-NEE). Our solution first generalizes MINE to estimate the entropy using a custom reference distribution. The entropy estimate can then be used to estimate the mutual information. We argue that the seemingly redundant intermediate step of entropy estimation allows one to improve the convergence by an appropriate reference distribution. In particular, we show that MI-NEE reduces to MINE in the special case when the reference distribution is the product of marginal distributions, but faster convergence is possible by choosing the uniform distribution as the reference distribution instead. Compared to the product of marginals, the uniform distribution introduces more samples in low-density regions and fewer samples in high-density regions, which appear to lead to an overall larger gradient for faster convergence.

cs.IT↗

arXiv2019

Multiterminal Secret Key Agreement with Nearly No Discussion

We consider the secret key agreement problem under the multiterminal source model proposed by Csiszár and Narayan. A single-letter characterization of the secrecy capacity is desired but remains unknown except in the extreme case with unlimited public discussion and without wiretapper's side information. Taking the problem to the opposite extreme by requiring the public discussion rate to be zero asymptotically, we obtain the desired characterization under surprisingly general setting with wiretapper's side information, silent users, trusted and untrusted helpers. An immediate consequence of the result is that the capacity with nearly no discussion is the same as the capacity with no discussion, resolving a previous conjecture in the affirmative. The idea of the proof is to characterize the capacity in the special case with neither wiretapper's side information nor untrusted helpers using a multivariate extension of Gács-Körner common information, and then extend the result to the general setting by a change of scenario that turns untrusted helpers into trusted helpers. We further show how to evaluate the capacity explicitly for finite linear sources and discuss how the current result can be extended to improve and unify existing bounds on the capacity for strictly positive discussion rates.

cs.IT↗

arXiv2019

One-Shot Perfect Secret Key Agreement for Finite Linear Sources

We consider a non-asymptotic (one-shot) version of the multiterminal secret key agreement problem on a finite linear source model. In this model, the observation of each terminal is a linear function of an underlying random vector composed of finitely many i.i.d. uniform random variables. Restricting the public discussion to be a linear function of the terminals' observations, we obtain a characterization of the communication complexity (minimum number of symbols of public discussion) of generating a secret key of maximum length. The minimum discussion is achieved by a non-interactive protocol in which each terminal first does a linear processing of its own private observations, following which the terminals all execute a discussion-optimal communication-for-omniscience protocol. The secret key is finally obtained as a linear function of the vector of all observations.

cs.IT↗

arXiv2018

Upper Bounds via Lamination on the Constrained Secrecy Capacity of Hypergraphical Sources

Hypergraphical sources are a natural class of sources for secret key generation, within which different subsets of terminals sharing secrets are allowed to discuss publicly in order to agree upon a global secret key. While their secrecy capacity, i.e., the maximum rate of a secret key that can be agreed upon by the entire set of terminals, is well-understood, what remains open is the maximum rate of a secret key that can be generated when there is a restriction on the overall rate of public discussion allowed. In this work, we obtain a family of explicitly computable upper bounds on the number of bits of secret key that can be generated per bit of public discussion. These upper bounds are derived using a lamination technique based on the submodularity of the entropy function. In particular, a specific instance of these upper bounds, called the edge-partition bound, is shown to be tight for the pairwise independent network model, a special case of the hypergraphical source. The secret key generation scheme achieving this upper bound is the tree-packing protocol of Nitinawarat et al., thereby resolving in the affirmative the discussion rate optimality of the tree packing protocol.

cs.IT↗

arXiv2018

Multiterminal Secret Key Agreement at Asymptotically Zero Discussion Rate

In the multiterminal secret key agreement problem, a set of users want to discuss with each other until they share a common secret key independent of their discussion. We want to characterize the maximum secret key rate, called the secrecy capacity, asymptotically when the total discussion rate goes to zero. In the case of only two users, the capacity is equal to the Gács-Körner common information. However, when there are more than two users, the capacity is unknown. It is plausible that a multivariate extension of the Gács-Körner common information is the capacity, however, proving the converse is challenging. We resolved this for the hypergraphical sources and finite linear sources, and provide efficiently computable characterizations. We also give some ideas of extending the techniques to more general source models.

cs.IT↗

arXiv2018

On the Optimality of Secret Key Agreement via Omniscience

For the multiterminal secret key agreement problem under a private source model, it is known that the maximum key rate, i.e., the secrecy capacity, can be achieved through communication for omniscience, but the omniscience strategy can be strictly suboptimal in terms of minimizing the public discussion rate. While a single-letter characterization is not known for the minimum discussion rate needed for achieving the secrecy capacity, we derive single-letter lower and upper bounds that yield some simple conditions for omniscience to be discussion-rate optimal. These conditions turn out to be enough to deduce the optimality of omniscience for a large class of sources including the hypergraphical sources. Through conjectures and examples, we explore other source models to which our methods do not easily extend.

cs.IT↗

arXiv2017

Compressed Secret Key Agreement: Maximizing Multivariate Mutual Information Per Bit

The multiterminal secret key agreement problem by public discussion is formulated with an additional source compression step where, prior to the public discussion phase, users independently compress their private sources to filter out strongly correlated components for generating a common secret key. The objective is to maximize the achievable key rate as a function of the joint entropy of the compressed sources. Since the maximum achievable key rate captures the total amount of information mutual to the compressed sources, an optimal compression scheme essentially maximizes the multivariate mutual information per bit of randomness of the private sources, and can therefore be viewed more generally as a dimension reduction technique. Single-letter lower and upper bounds on the maximum achievable key rate are derived for the general source model, and an explicit polynomial-time computable formula is obtained for the pairwise independent network model. In particular, the converse results and the upper bounds are obtained from those of the related secret key agreement problem with rate-limited discussion. A precise duality is shown for the two-user case with one-way discussion, and such duality is extended to obtain the desired converse results in the multi-user case. In addition to posing new challenges in information processing and dimension reduction, the compressed secret key agreement problem helps shed new light on resolving the difficult problem of secret key agreement with rate-limited discussion, by offering a more structured achieving scheme and some simpler conjectures to prove.

cs.IT↗

arXiv2017

Agglomerative Info-Clustering

An agglomerative clustering of random variables is proposed, where clusters of random variables sharing the maximum amount of multivariate mutual information are merged successively to form larger clusters. Compared to the previous info-clustering algorithms, the agglomerative approach allows the computation to stop earlier when clusters of desired size and accuracy are obtained. An efficient algorithm is also derived based on the submodularity of entropy and the duality between the principal sequence of partitions and the principal sequence for submodular functions.

cs.IT↗

arXiv2017

Info-Clustering: An Efficient Algorithm by Network Information Flow

Motivated by the fact that entities in a social network or biological system often interact by exchanging information, we propose an efficient info-clustering algorithm that can group entities into communities using a parametric max-flow algorithm. This is a meaningful special case of the info-clustering paradigm where the dependency structure is graphical and can be learned readily from data.

cs.IT↗

arXiv2017

Secret Key Agreement under Discussion Rate Constraints

For the multiterminal secret key agreement problem, new single-letter lower bounds are obtained on the public discussion rate required to achieve any given secret key rate below the secrecy capacity. The results apply to general source model without helpers or wiretapper's side information but can be strengthened for hypergraphical sources. In particular, for the pairwise independent network, the results give rise to a complete characterization of the maximum secret key rate achievable under a constraint on the total discussion rate.

cs.IT↗

arXiv2016

Info-Clustering: A Mathematical Theory for Data Clustering

We formulate an info-clustering paradigm based on a multivariate information measure, called multivariate mutual information, that naturally extends Shannon's mutual information between two random variables to the multivariate case involving more than two random variables. With proper model reductions, we show that the paradigm can be applied to study the human genome and connectome in a more meaningful way than the conventional algorithmic approach. Not only can info-clustering provide justifications and refinements to some existing techniques, but it also inspires new computationally feasible solutions.

cs.IT↗

arXiv2016

Duality between Feature Selection and Data Clustering

The feature-selection problem is formulated from an information-theoretic perspective. We show that the problem can be efficiently solved by an extension of the recently proposed info-clustering paradigm. This reveals the fundamental duality between feature selection and data clustering,which is a consequence of the more general duality between the principal partition and the principal lattice of partitions in combinatorial optimization.

cs.IT↗