Searcharxiv⌕ Search

arXiv subjects

Yaming Yu

Publications and source records attributed to Yaming Yu.

35 records · Page 2Linked to original sources
arXiv2010

Monotonicity, thinning and discrete versions of the Entropy Power Inequality

We consider the entropy of sums of independent discrete random variables, in analogy with Shannon's Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for Poisson variables. We show that some natural analogues of the Entropy Power Inequality do not in fact hold, but propose an alternative formulation which does always hold. The key to many proofs of Shannon's Entropy Power Inequality is the behaviour of entropy on scaling of continuous random variables. We believe that Rényi's operation of thinning discrete random variables plays a similar role to scaling, and give a sharp bound on how the entropy of ultra log-concave random variables behaves on thinning. In the spirit of the monotonicity results established by Artstein, Ball, Barthe and Naor, we prove a stronger version of concavity of entropy, which implies a strengthened form of our discrete Entropy Power Inequality.

cs.IT↗
arXiv2010

D-optimal designs via a cocktail algorithm

A fast new algorithm is proposed for numerical computation of (approximate) D-optimal designs. This "cocktail algorithm" extends the well-known vertex direction method (VDM; Fedorov 1972) and the multiplicative algorithm (Silvey, Titterington and Torsney, 1978), and shares their simplicity and monotonic convergence properties. Numerical examples show that the cocktail algorithm can lead to dramatically improved speed, sometimes by orders of magnitude, relative to either the multiplicative algorithm or the vertex exchange method (a variant of VDM). Key to the improved speed is a new nearest neighbor exchange strategy, which acts locally and complements the global effect of the multiplicative algorithm. Possible extensions to related problems such as nonparametric maximum likelihood estimation are mentioned.

stat.CO↗
arXiv2010

Improved EM for Mixture Proportions with Applications to Nonparametric ML Estimation for Censored Data

Improved EM strategies, based on the idea of efficient data augmentation (Meng and van Dyk 1997, 1998), are presented for ML estimation of mixture proportions. The resulting algorithms inherit the simplicity, ease of implementation, and monotonic convergence properties of EM, but have considerably improved speed. Because conventional EM tends to be slow when there exists a large overlap between the mixture components, we can improve the speed without sacrificing the simplicity or stability, if we can reformulate the problem so as to reduce the amount of overlap. We propose simple "squeezing" strategies for that purpose. Moreover, for high-dimensional problems, such as computing the nonparametric MLE of the distribution function with censored data, a natural and effective remedy for conventional EM is to add exchange steps (based on improved EM) between adjacent mixture components, where the overlap is most severe. Theoretical considerations show that the resulting EM-type algorithms, when carefully implemented, are globally convergent. Simulated and real data examples show dramatic improvement in speed in realistic situations.

stat.CO↗
arXiv2010

Sharp Bounds on the Entropy of the Poisson Law and Related Quantities

One of the difficulties in calculating the capacity of certain Poisson channels is that H(lambda), the entropy of the Poisson distribution with mean lambda, is not available in a simple form. In this work we derive upper and lower bounds for H(lambda) that are asymptotically tight and easy to compute. The derivation of such bounds involves only simple probabilistic and analytic tools. This complements the asymptotic expansions of Knessl (1998), Jacquet and Szpankowski (1999), and Flajolet (1999). The same method yields tight bounds on the relative entropy D(n, p) between a binomial and a Poisson, thus refining the work of Harremoes and Ruzankin (2004). Bounds on the entropy of the binomial also follow easily.

cs.IT↗
arXiv2009

Stochastic Ordering of Exponential Family Distributions and Their Mixtures

We investigate stochastic comparisons between exponential family distributions and their mixtures with respect to the usual stochastic order, the hazard rate order, the reversed hazard rate order, and the likelihood ratio order. A general theorem based on the notion of relative log-concavity is shown to unify various specific results for the Poisson, binomial, negative binomial, and gamma distributions in recent literature. By expressing a convolution of gamma distributions with arbitrary scale and shape parameters as a scale mixture of gamma distributions, we obtain comparison theorems concerning such convolutions that generalize some known results. Analogous results on convolutions of negative binomial distributions are also discussed.

math.ST↗
arXiv2009

Efficient Simulation of a Bivariate Exponential Conditionals Distribution

The bivariate distribution with exponential conditionals (BEC) is introduced by Arnold and Strauss [Bivariate distributions with exponential conditionals, J. Amer. Statist. Assoc. 83 (1988) 522--527]. This work presents a simple and fast algorithm for simulating random variates from this density.

stat.CO↗
arXiv2009

Squeezing the Arimoto-Blahut algorithm for faster convergence

The Arimoto--Blahut algorithm for computing the capacity of a discrete memoryless channel is revisited. A so-called ``squeezing'' strategy is used to design algorithms that preserve its simplicity and monotonic convergence properties, but have provably better rates of convergence.

cs.IT↗
arXiv2009

Monotonic Convergence in an Information-Theoretic Law of Small Numbers

An "entropy increasing to the maximum" result analogous to the entropic central limit theorem (Barron 1986; Artstein et al. 2004) is obtained in the discrete setting. This involves the thinning operation and a Poisson limit. Monotonic convergence in relative entropy is established for general discrete distributions, while monotonic increase of Shannon entropy is proved for the special class of ultra-log-concave distributions. Overall we extend the parallel between the information-theoretic central limit theorem and law of small numbers explored by Kontoyiannis et al. (2005) and Harremoës et al.\ (2007, 2008). Ingredients in the proofs include convexity, majorization, and stochastic orders.

cs.IT↗
arXiv2009

Concavity of entropy under thinning

Building on the recent work of Johnson (2007) and Yu (2008), we prove that entropy is a concave function with respect to the thinning operation T_a. That is, if X and Y are independent random variables on Z_+ with ultra-log-concave probability mass functions, then H(T_a X+T_{1-a} Y)>= a H(X)+(1-a)H(Y), 0 <= a <= 1, where H denotes the discrete entropy. This is a discrete analogue of the inequality (h denotes the differential entropy) h(sqrt(a) X + sqrt{1-a} Y)>= a h(X)+(1-a) h(Y), 0 <= a <= 1, which holds for continuous X and Y with finite variances and is equivalent to Shannon's entropy power inequality. As a consequence we establish a special case of a conjecture of Shepp and Olkin (1981).

cs.IT↗
arXiv2009

On an Asymptotic Series of Ramanujan

An asymptotic series in Ramanujan's second notebook (Entry 10, Chapter 3) is concerned with the behavior of the expected value of $ϕ(X)$ for large $λ$ where $X$ is a Poisson random variable with mean $λ$ and $ϕ$ is a function satisfying certain growth conditions. We generalize this by studying the asymptotics of the expected value of $ϕ(X)$ when the distribution of $X$ belongs to a suitable family indexed by a convolution parameter. Examples include the problem of inverse moments for distribution families such as the binomial or the negative binomial.

math.ST↗
arXiv2009

Confirming Two Conjectures of Su and Wang

Two conjectures of Su and Wang (2008) concerning binomial coefficients are proved. For $n\geq k\geq 0$ and $b>a>0$, we show that the finite sequence $C_j=\binom{n+ja}{k+jb}$ is a Pólya frequency sequence. For $n\geq k\geq 0$ and $a>b>0$, we show that there exists an integer $m\geq 0$ such that the infinite sequence $\binom{n+ja}{k+jb}, j=0, 1,...$, is log-concave for $0\leq j\leq m$ and log-convex for $j\geq m$. The proof of the first result exploits the connection between total positivity and planar networks, while that of the second uses a variation-diminishing property of the Laplace transform.

math.CO↗