arXiv subjects

Hongyi Yao

Publications and source records attributed to Hongyi Yao.

12 recordsLinked to original sources

arXiv2022

RAPQ: Rescuing Accuracy for Power-of-Two Low-bit Post-training Quantization

We introduce a Power-of-Two low-bit post-training quantization(PTQ) method for deep neural network that meets hardware requirements and does not call for long-time retraining. Power-of-Two quantization can convert the multiplication introduced by quantization and dequantization to bit-shift that is adopted by many efficient accelerators. However, the Power-of-Two scale factors have fewer candidate values, which leads to more rounding or clipping errors. We propose a novel Power-of-Two PTQ framework, dubbed RAPQ, which dynamically adjusts the Power-of-Two scales of the whole network instead of statically determining them layer by layer. It can theoretically trade off the rounding error and clipping error of the whole network. Meanwhile, the reconstruction method in RAPQ is based on the BN information of every unit. Extensive experiments on ImageNet prove the excellent performance of our proposed method. Without bells and whistles, RAPQ can reach accuracy of 65% and 48% on ResNet-18 and MobileNetV2 respectively with weight INT2 activation INT4. We are the first to propose the more constrained but hardware-friendly Power-of-Two quantization scheme for low-bit PTQ specially and prove that it can achieve nearly the same accuracy as SOTA PTQ method. The code was released.

cs.CV↗

arXiv2021

AdaPruner: Adaptive Channel Pruning and Effective Weights Inheritance

Channel pruning is one of the major compression approaches for deep neural networks. While previous pruning methods have mostly focused on identifying unimportant channels, channel pruning is considered as a special case of neural architecture search in recent years. However, existing methods are either complicated or prone to sub-optimal pruning. In this paper, we propose a pruning framework that adaptively determines the number of each layer's channels as well as the wights inheritance criteria for sub-network. Firstly, evaluate the importance of each block in the network based on the mean of the scaling parameters of the BN layers. Secondly, use the bisection method to quickly find the compact sub-network satisfying the budget. Finally, adaptively and efficiently choose the weight inheritance criterion that fits the current architecture and fine-tune the pruned network to recover performance. AdaPruner allows to obtain pruned network quickly, accurately and efficiently, taking into account both the structure and initialization weights. We prune the currently popular CNN models (VGG, ResNet, MobileNetV2) on different image classification datasets, and the experimental results demonstrate the effectiveness of our proposed method. On ImageNet, we reduce 32.8% FLOPs of MobileNetV2 with only 0.62% decrease for top-1 accuracy, which exceeds all previous state-of-the-art channel pruning methods. The code will be released.

cs.CV↗

arXiv2013

Distributed Reed-Solomon Codes for Simple Multiple Access Networks

We consider a simple multiple access network in which a destination node receives information from multiple sources via a set of relay nodes. Each relay node has access to a subset of the sources, and is connected to the destination by a unit capacity link. We also assume that $z$ of the relay nodes are adversarial. We propose a computationally efficient distributed coding scheme and show that it achieves the full capacity region for up to three sources. Specifically, the relay nodes encode in a distributed fashion such that the overall codewords received at the destination are codewords from a single Reed-Solomon code.

cs.IT↗

arXiv2013

Rateless Resilient Network Coding Against Byzantine Adversaries

This paper considers rateless network error correction codes for reliable multicast in the presence of adversarial errors. Most existing network error correction codes are designed for a given network capacity and maximum number of errors known a priori to the encoder and decoder. However, in certain practical settings it may be necessary to operate without such a priori knowledge. We present rateless coding schemes for two adversarial models, where the source sends more redundancy over time, until decoding succeeds. The first model assumes there is a secret channel between the source and the destination that the adversaries cannot overhear. The rate of the channel is negligible compared to the main network. In the second model, instead of a secret channel, the source and destination share random secrets independent of the input information. The amount of secret information required is negligible compared to the amount of information sent. Both schemes are optimal in that decoding succeeds with high probability when the total amount of information received by the sink satisfies the cut set bound with respect to the amount of message and error information. The schemes are distributed, polynomial-time and end-to-end in that other than the source and destination nodes, other intermediate nodes carry out classical random linear network coding.

cs.IT↗

arXiv2012

Network Equivalence in the Presence of an Eavesdropper

We consider networks of noisy degraded wiretap channels in the presence of an eavesdropper. For the case where the eavesdropper can wiretap at most one channel at a time, we show that the secrecy capacity region, for a broad class of channels and any given network topology and communication demands, is equivalent to that of a corresponding network where each noisy wiretap channel is replaced by a noiseless wiretap channel. Thus in this case there is a separation between wiretap channel coding on each channel and secure network coding on the resulting noiseless network. We show with an example that such separation does not hold when the eavesdropper can access multiple channels at the same time, for which case we provide upper and lower bounding noiseless networks.

cs.IT↗

arXiv2011

Low-Complexity Near-Optimal Codes for Gaussian Relay Networks

We consider the problem of information flow over Gaussian relay networks. Similar to the recent work by Avestimehr \emph{et al.} [1], we propose network codes that achieve up to a constant gap from the capacity of such networks. However, our proposed codes are also computationally tractable. Our main technique is to use the codes of Avestimehr \emph{et al.} as inner codes in a concatenated coding scheme.

cs.IT↗

arXiv2010

Multiple-access Network Information-flow and Correction Codes

This work considers the multiple-access multicast error-correction scenario over a packetized network with $z$ malicious edge adversaries. The network has min-cut $m$ and packets of length $\ell$, and each sink demands all information from the set of sources $\sources$. The capacity region is characterized for both a "side-channel" model (where sources and sinks share some random bits that are secret from the adversary) and an "omniscient" adversarial model (where no limitations on the adversary's knowledge are assumed). In the "side-channel" adversarial model, the use of a secret channel allows higher rates to be achieved compared to the "omniscient" adversarial model, and a polynomial-complexity capacity-achieving code is provided. For the "omniscient" adversarial model, two capacity-achieving constructions are given: the first is based on random subspace code design and has complexity exponential in $\ell m$, while the second uses a novel multiple-field-extension technique and has $O(\ell m^{|\sources|})$ complexity, which is polynomial in the network size. Our code constructions are "end-to-end" in that all nodes except the sources and sinks are oblivious to the adversaries and may simply implement predesigned linear network codes (random or otherwise). Also, the sources act independently without knowledge of the data from other sources.

cs.IT↗

arXiv2010

Achieving the Scaling Law of SNR-Monitoring in Dynamic Wireless Networks

The characteristics of wireless communication channels may vary with time due to fading, environmental changes and movement of mobile wireless devices. Tracking and estimating channel gains of wireless channels is therefore a fundamentally important element of many wireless communication systems. In particular, the receivers in many wireless networks need to estimate the channel gains by means of a training sequence. This paper studies the scaling law (on the network size) of the overhead for channel gain monitoring in wireless network. We first investigate the scenario in which a receiver needs to track the channel gains with respect to multiple transmitters. To be concrete, suppose that there are n transmitters, and that in the current round of channel-gain estimation, no more than k channels suffer significant variations since the last round. We proves that "Θ(k\log((n+1)/k)) time slots" is the minimum number of time slots needed to catch up with the k varied channels. At the same time, we propose a novel channel-gain monitoring scheme named ADMOT to achieve the overhead lower-bound. ADMOT leverages recent advances in compressive sensing in signal processing and interference processing in wireless communication, to enable the receiver to estimate all n channels in a reliable and computationally efficient manner within O(k\log((n+1)/k)) time slots. To our best knowledge, all previous channel-tracking schemes require Θ(n) time slots regardless of k. Note that based on above results for single receiver scenario, the scaling law of general setting is achieved in which there are multiple transmitters, relay nodes and receivers.

cs.NI↗

arXiv2010

Network Codes Resilient to Jamming and Eavesdropping

We consider the problem of communicating information over a network secretly and reliably in the presence of a hidden adversary who can eavesdrop and inject malicious errors. We provide polynomial-time, rate-optimal distributed network codes for this scenario, improving on the rates achievable in previous work. Our main contribution shows that as long as the sum of the adversary's jamming rate Zo and his eavesdropping rate Zi is less than the network capacity C, (i.e., Zo+Zi<C), our codes can communicate (with vanishingly small error probability) a single bit correctly and without leaking any information to the adversary. We then use this to design codes that allow communication at the optimal source rate of C-Zo-Zi, while keeping the communicated message secret from the adversary. Interior nodes are oblivious to the presence of adversaries and perform random linear network coding; only the source and destination need to be tweaked. In proving our results we correct an error in prior work by a subset of the authors in this work.

cs.NI↗

arXiv2010

Passive network tomography for erroneous networks: A network coding approach

Passive network tomography uses end-to-end observations of network communication to characterize the network, for instance to estimate the network topology and to localize random or adversarial glitches. Under the setting of linear network coding this work provides a comprehensive study of passive network tomography in the presence of network (random or adversarial) glitches. To be concrete, this work is developed along two directions: 1. Tomographic upper and lower bounds (i.e., the most adverse conditions in each problem setting under which network tomography is possible, and corresponding schemes (computationally efficient, if possible) that achieve this performance) are presented for random linear network coding (RLNC). We consider RLNC designed with common randomness, i.e., the receiver knows the random code-books all nodes. (To justify this, we show an upper bound for the problem of topology estimation in networks using RLNC without common randomness.) In this setting we present the first set of algorithms that characterize the network topology exactly. Our algorithm for topology estimation with random network errors has time complexity that is polynomial in network parameters. For the problem of network error localization given the topology information, we present the first computationally tractable algorithm to localize random errors, and prove it is computationally intractable to localize adversarial errors. 2. New network coding schemes are designed that improve the tomographic performance of RLNC while maintaining the desirable low-complexity, throughput-optimal, distributed linear network coding properties of RLNC. In particular, we design network codes based on Reed-Solomon codes so that a maximal number of adversarial errors can be localized in a computationally efficient manner even without the information of network topology.

cs.NI↗

arXiv2010

Multiple Access Network Information-flow And Correction codes

The network communication scenario where one or more receivers request all the information transmitted by different sources is considered. We introduce distributed polynomial-time network codes in the presence of malicious nodes. Our codes can achieve any point inside the rate region of multiple-source multicast transmission scenarios both in the cases of coherent and non-coherent network coding. For both cases the encoding and decoding algorithm runs in poly(|E|)exp(s) time, where poly(|E|) is a polynomial function of the number of edges |E| in the network and exp(s) is an exponential function of the number of sources s. Our codes are fully distributed and different sources require no knowledge of the data transmitted by their peers. Our codes are "end-to-end", that is, all nodes apart from the sources and the receivers are oblivious to the adversaries present in the network and simply implement random linear network coding.

cs.NI↗

arXiv2007

On the Information of the Second Moments Between Random Variables Using Mutually Unbiased Bases

The notation of mutually unbiased bases(MUB) was first introduced by Ivanovic to reconstruct density matrixes\cite{Ivanovic}. The subject about how to use MUB to analyze, process, and utilize the information of the second moments between random variables is studied in this paper. In the first part, the mathematical foundation will be built. It will be shown that the spectra of MUB have complete information for the correlation matrixes of finite discrete signals, and the nice properties of them. Roughly speaking, it will be shown that each spectrum from MUB plays an equal role for finite discrete signals, and the effect between any two spectra can be treated as a global constant shift. These properties will be used to find some important and natural characterizations of random vectors and random discrete operators/filters. For a technical reason, it will be shown that any MUB spectra can be found as fast as Fourier spectrum when the length of the signal is a prime number. In the second part, some applications will be presented. First of all, a protocol about how to increase the number of users in a basic digital communication model will be studied, which has bring some deep insights about how to encode the information into the second moments between random variables. Secondly, the application of signal analysis will be studied. It is suggested that complete "MUB" spectra analysis works well in any case, and people can just choose the spectra they are interested in to do analysis. For instance, single Fourier spectra analysis can be also applied in nonstationary case. Finally, the application of MUB in dimensionality reduction will be considered, when the prior knowledge of the data isn't reliable.

cs.IT↗