arXiv subjects

Quan Geng

Publications and source records attributed to Quan Geng.

10 recordsLinked to original sources

arXiv2020

Accelerating Large-Scale Inference with Anisotropic Vector Quantization

Quantization based techniques are the current state-of-the-art for scaling maximum inner product search to massive databases. Traditional approaches to quantization aim to minimize the reconstruction error of the database points. Based on the observation that for a given query, the database points that have the largest inner products are more relevant, we develop a family of anisotropic quantization loss functions. Under natural statistical assumptions, we show that quantization with these loss functions leads to a new variant of vector quantization that more greatly penalizes the parallel component of a datapoint's residual relative to its orthogonal component. The proposed approach achieves state-of-the-art results on the public benchmarks available at \url{ann-benchmarks.com}.

cs.LG↗

arXiv2019

On the Capacity Region of Broadcast Packet Erasure Relay Networks With Feedback

We derive a new outer bound on the capacity region of broadcast traffic in multiple input broadcast packet erasure channels with feedback, and extend this outer bound to packet erasure relay networks with feedback. We show the tightness of the outer bound for various classes of networks. An important engineering implication of this work is that for network coding schemes for parallel broadcast channels, the `xor' packets should be sent over correlated broadcast subchannels.

cs.IT↗

arXiv2019

Optimal Noise-Adding Mechanism in Additive Differential Privacy

We derive the optimal $(0, δ)$-differentially private query-output independent noise-adding mechanism for single real-valued query function under a general cost-minimization framework. Under a mild technical condition, we show that the optimal noise probability distribution is a uniform distribution with a probability mass at the origin. We explicitly derive the optimal noise distribution for general $\ell^p$ cost functions, including $\ell^1$ (for noise magnitude) and $\ell^2$ (for noise power) cost functions, and show that the probability concentration on the origin occurs when $δ> \frac{p}{p+1}$. Our result demonstrates an improvement over the existing Gaussian mechanisms by a factor of two and three for $(0,δ)$-differential privacy in the high privacy regime in the context of minimizing the noise magnitude and noise power, and the gain is more pronounced in the low privacy regime. Our result is consistent with the existing result for $(0,δ)$-differential privacy in the discrete setting, and identifies a probability concentration phenomenon in the continuous setting.

cs.CR↗

arXiv2019

Privacy and Utility Tradeoff in Approximate Differential Privacy

We characterize the minimum noise amplitude and power for noise-adding mechanisms in $(ε, δ)$-differential privacy for single real-valued query function. We derive new lower bounds using the duality of linear programming, and new upper bounds by proposing a new class of $(ε,δ)$-differentially private mechanisms, the \emph{truncated Laplacian} mechanisms. We show that the multiplicative gap of the lower bounds and upper bounds goes to zero in various high privacy regimes, proving the tightness of the lower and upper bounds and thus establishing the optimality of the truncated Laplacian mechanism. In particular, our results close the previous constant multiplicative gap in the discrete setting. Numeric experiments show the improvement of the truncated Laplacian mechanism over the optimal Gaussian mechanism in all privacy regimes.

cs.CR↗

arXiv2014

Symmetric Two-User Gaussian Interference Channel with Common Messages

We consider symmetric two-user Gaussian interference channel with common messages. We derive an upper bound on the sum capacity, and show that the upper bound is tight in the low interference regime, where the optimal transmission scheme is to send no common messages and each receiver treats interference as noise. Our result shows that although the availability of common messages provides a cooperation opportunity for transmitters, in the low interference regime the presence of common messages does not help increase the sum capacity.

cs.IT↗

arXiv2013

Optimal Noise Adding Mechanisms for Approximate Differential Privacy

We study the (nearly) optimal mechanisms in $(ε,δ)$-approximate differential privacy for integer-valued query functions and vector-valued (histogram-like) query functions under a utility-maximization/cost-minimization framework. We characterize the tradeoff between $ε$ and $δ$ in utility and privacy analysis for histogram-like query functions ($\ell^1$ sensitivity), and show that the $(ε,δ)$-differential privacy is a framework not much more general than the $(ε,0)$-differential privacy and $(0,δ)$-differential privacy in the context of $\ell^1$ and $\ell^2$ cost functions, i.e., minimum expected noise magnitude and noise power. In the same context of $\ell^1$ and $\ell^2$ cost functions, we show the near-optimality of uniform noise mechanism and discrete Laplacian mechanism in the high privacy regime (as $(ε,δ) \to (0,0)$). We conclude that in $(ε,δ)$-differential privacy, the optimal noise magnitude and noise power are $Θ(\min(\frac{1}ε,\frac{1}δ))$ and $Θ(\min(\frac{1}{ε^2},\frac{1}{δ^2}))$, respectively, in the high privacy regime.

cs.DS↗

arXiv2013

The Optimal Mechanism in Differential Privacy: Multidimensional Setting

We derive the optimal $ε$-differentially private mechanism for a general two-dimensional real-valued (histogram-like) query function under a utility-maximization (or cost-minimization) framework for the $\ell^1$ cost function. We show that the optimal noise probability distribution has a correlated multidimensional staircase-shaped probability density function. Compared with the Laplacian mechanism, we show that in the high privacy regime (as $ε\to 0$), the Laplacian mechanism is approximately optimal; and in the low privacy regime (as $ε\to +\infty$), the optimal cost is $Θ(e^{-\fracε{3}})$, while the cost of the Laplacian mechanism is $\frac{2Δ}ε$, where $Δ$ is the sensitivity of the query function. We conclude that the gain is more pronounced in the low privacy regime. We conjecture that the optimality of the staircase mechanism holds for vector-valued (histogram-like) query functions with arbitrary dimension, and holds for many other classes of cost functions as well.

cs.CR↗

arXiv2013

The Optimal Mechanism in Differential Privacy

We derive the optimal $ε$-differentially private mechanism for single real-valued query function under a very general utility-maximization (or cost-minimization) framework. The class of noise probability distributions in the optimal mechanism has {\em staircase-shaped} probability density functions which are symmetric (around the origin), monotonically decreasing and geometrically decaying. The staircase mechanism can be viewed as a {\em geometric mixture of uniform probability distributions}, providing a simple algorithmic description for the mechanism. Furthermore, the staircase mechanism naturally generalizes to discrete query output settings as well as more abstract settings. We explicitly derive the optimal noise probability distributions with minimum expectation of noise amplitude and power. Comparing the optimal performances with those of the Laplacian mechanism, we show that in the high privacy regime ($ε$ is small), Laplacian mechanism is asymptotically optimal as $ε\to 0$; in the low privacy regime ($ε$ is large), the minimum expectation of noise amplitude and minimum noise power are $Θ(Δe^{-\fracε{2}})$ and $Θ(Δ^2 e^{-\frac{2ε}{3}})$ as $ε\to +\infty$, while the expectation of noise amplitude and power using the Laplacian mechanism are $\fracΔε$ and $\frac{2Δ^2}{ε^2}$, where $Δ$ is the sensitivity of the query function. We conclude that the gains are more pronounced in the low privacy regime.

cs.CR↗

arXiv2013

Interactive Interference Alignment

We study interference channels (IFC) where interaction among sources and destinations is enabled, e.g., both sources and destinations can talk to each other using full-duplex radios. The interaction can come in two ways: 1) {\em In-band interaction:} sources and destinations can transmit and listen in the same channel simultaneously, enabling interaction. 2) {\em out-of-band interaction:} destinations talk back to the sources on an out-of-band channel, possible from white-space channels. The flexibility afforded by interaction among sources and destinations allows for the derivation of interference alignment (IA) strategies that have desirable "engineering properties": insensitivity to the rationality or irrationality of channel parameters, small block lengths and finite SNR operations. We show that for several classes of interference channels the interactive interference alignment scheme can achieve the optimal degrees of freedom. In particular, we show the {\em first simple scheme} (having finite block length, for channels having no diversity) for $K=3,4$ that can achieve the optimal degrees of freedom of $\frac{K}{2}$ even after accounting for the cost of interaction. We also give simulation results on the finite SNR performance of interactive alignment under some settings. On the technical side, we show using a Gröbner basis argument that in a general network potentially utilizing cooperation and feedback, the optimal degrees of freedom under linear schemes of a fixed block length is the same for channel coefficients with probability 1. Furthermore, a numerical method to estimate this value is also presented. These tools have potentially wider utility in studying other wireless networks as well.

cs.IT↗

arXiv2011

On the Local Correctness of L^1 Minimization for Dictionary Learning

The idea that many important classes of signals can be well-represented by linear combinations of a small set of atoms selected from a given dictionary has had dramatic impact on the theory and practice of signal processing. For practical problems in which an appropriate sparsifying dictionary is not known ahead of time, a very popular and successful heuristic is to search for a dictionary that minimizes an appropriate sparsity surrogate over a given set of sample data. While this idea is appealing, the behavior of these algorithms is largely a mystery; although there is a body of empirical evidence suggesting they do learn very effective representations, there is little theory to guarantee when they will behave correctly, or when the learned dictionary can be expected to generalize. In this paper, we take a step towards such a theory. We show that under mild hypotheses, the dictionary learning problem is locally well-posed: the desired solution is indeed a local minimum of the $\ell^1$ norm. Namely, if $\mb A \in \Re^{m \times n}$ is an incoherent (and possibly overcomplete) dictionary, and the coefficients $\mb X \in \Re^{n \times p}$ follow a random sparse model, then with high probability $(\mb A,\mb X)$ is a local minimum of the $\ell^1$ norm over the manifold of factorizations $(\mb A',\mb X')$ satisfying $\mb A' \mb X' = \mb Y$, provided the number of samples $p = Ω(n^3 k)$. For overcomplete $\mb A$, this is the first result showing that the dictionary learning problem is locally solvable. Our analysis draws on tools developed for the problem of completing a low-rank matrix from a small subset of its entries, which allow us to overcome a number of technical obstacles; in particular, the absence of the restricted isometry property.

cs.IT↗