arXiv subjects

Tomohiro Nishiyama

Publications and source records attributed to Tomohiro Nishiyama.

33 records · Page 2Linked to original sources

arXiv2022

Thermodynamic-kinetic uncertainty relation: properties and an information-theoretic interpretation

Universal relations that characterize the fluctuations of nonequilibrium systems are of fundamental importance. The thermodynamic and kinetic uncertainty relations impose upper bounds on the precision of currents solely by total entropy production and dynamical activity, respectively. Recently, a tighter bound that imposes on the precision of currents by both total entropy production and dynamical activity has been derived (referred to as the TKUR). In this paper, we show that the TKUR gives the tightest bound of a class of inequalities that imposes an upper bound on the precision of currents by arbitrary functions of the entropy production, dynamical activity, and time interval. Furthermore, we show that the TKUR can be rewritten as an inequality between two Kullback-Leibler divergences. One comes from the ratio of entropy production to dynamical activity, the other comes from the Kullback-Leibler divergence between two probability distributions defined on two-element set, which are characterized by the ratio of precision of the time-integrated current to dynamical activity.

cond-mat.stat-mech↗

arXiv2021

A Tight Lower Bound for the Hellinger Distance with Given Means and Variances

The binary divergences that are divergences between probability measures defined on the same 2-point set have an interesting property. For the chi-squared divergence and the relative entropy, it is known that their binary divergence attain lower bounds with given means and variances, respectively. In this note, we show that the binary divergence of the squared Hellinger distance has the same property and propose an open problem that what conditions are needed for f-divergence to satisfy this property.

cs.IT↗

arXiv2020

On Relations Between the Relative entropy and $χ^2$-Divergence, Generalizations and Applications

The relative entropy and chi-squared divergence are fundamental divergence measures in information theory and statistics. This paper is focused on a study of integral relations between the two divergences, the implications of these relations, their information-theoretic applications, and some generalizations pertaining to the rich class of $f$-divergences. Applications that are studied in this paper refer to lossless compression, the method of types and large deviations, strong~data-processing inequalities, bounds on contraction coefficients and maximal correlation, and the convergence rate to stationarity of a type of discrete-time Markov chains.

cs.IT↗

arXiv2020

Convex Optimization on Functionals of Probability Densities

In information theory, some optimization problems result in convex optimization problems on strictly convex functionals of probability densities. In this note, we study these problems and show conditions of minimizers and the uniqueness of the minimizer if there exist a minimizer.

cs.IT↗

arXiv2020

Minimization Problems on Strictly Convex Divergences

The divergence minimization problem plays an important role in various fields. In this note, we focus on differentiable and strictly convex divergences. For some minimization problems, we show the minimizer conditions and the uniqueness of the minimizer without assuming a specific form of divergences. Furthermore, we show geometric properties related to the minimization problems.

cs.IT↗

arXiv2019

A New Lower Bound for Kullback-Leibler Divergence Based on Hammersley-Chapman-Robbins Bound

In this paper, we derive a useful lower bound for the Kullback-Leibler divergence (KL-divergence) based on the Hammersley-Chapman-Robbins bound (HCRB). The HCRB states that the variance of an estimator is bounded from below by the Chi-square divergence and the expectation value of the estimator. By using the relation between the KL-divergence and the Chi-square divergence, we show that the lower bound for the KL-divergence which only depends on the expectation value and the variance of a function we choose. This lower bound can also be derived from an information geometric approach. Furthermore, we show that the equality holds for the Bernoulli distributions and show that the inequality converges to the Cramér-Rao bound when two distributions are very close. We also describe application examples and examples of numerical calculation.

math.ST↗

arXiv2019

Monotonically Decreasing Sequence of Divergences

Divergences are quantities that measure discrepancy between two probability distributions and play an important role in various fields such as statistics and machine learning. Divergences are non-negative and are equal to zero if and only if two distributions are the same. In addition, some important divergences such as the f-divergence have convexity, which we call "convex divergence". In this paper, we show new properties of the convex divergences by using integral and differential operators that we introduce. For the convex divergence, the result applied the integral or differential operator is also a divergence. In particular, the integral operator preserves convexity. Furthermore, the results applied the integral operator multiple times constitute a monotonically decreasing sequence of the convex divergences. We derive new sequences of the convex divergences that include the Kullback-Leibler divergence or the reverse Kullback-Leibler divergence from these properties.

math.ST↗

arXiv2019

Cramér-Rao-type Bound and Stam's Inequality for Discrete Random Variables

The variance and the entropy power of a continuous random variable are bounded from below by the reciprocal of its Fisher information through the Cramér-Rao bound and the Stam's inequality respectively. In this note, we introduce the Fisher information for discrete random variables and derive the discrete Cramér-Rao-type bound and the discrete Stam's inequality.

math.ST↗

arXiv2019

Inequalities between $L^p$-norms for log-concave distributions

Log-concave distributions include some important distributions such as normal distribution, exponential distribution and so on. In this note, we show inequalities between two Lp-norms for log-concave distributions on the Euclidean space. These inequalities are the generalizations of the upper and lower bound of the differential entropy and are also interpreted as a kind of expansion of the inequality between two Lp-norms on the measurable set with finite measure.

math.ST↗

arXiv2019

$L^p$-norm inequality using q-moment and its applications

For a measurable function on a set which has a finite measure, an inequality holds between two Lp-norms. In this paper, we show similar inequalities for the Euclidean space and the Lebesgue measure by using a q-moment which is a moment of an escort distribution. As applications of these inequalities, we first derive upper bounds for the Renyi and the Tsallis entropies with given q-moment and derive an inequality between two Renyi entropies. Second, we derive an upper bound for the probability of a subset in the Euclidean space with given Lp-norm on the same set.

math.ST↗

arXiv2018

Divergence Network: Graphical calculation method of divergence functions

In this paper, we introduce directed networks called `divergence network' in order to perform graphical calculation of divergence functions. By using the divergence networks, we can easily understand the geometric meaning of calculation results and grasp relations among divergence functions intuitively.

cs.LG↗

arXiv2018

Sum decomposition of divergence into three divergences

Divergence functions play a key role as to measure the discrepancy between two points in the field of machine learning, statistics and signal processing. Well-known divergences are the Bregman divergences, the Jensen divergences and the f-divergences. In this paper, we show that the symmetric Bregman divergence can be decomposed into the sum of two types of Jensen divergences and the Bregman divergence. Furthermore, applying this result, we show another sum decomposition of divergence is possible which includes f-divergences explicitly.

stat.ME↗

arXiv2018

Generalized Bregman and Jensen divergences which include some f-divergences

In this paper, we introduce new classes of divergences by extending the definitions of the Bregman divergence and the skew Jensen divergence. These new divergence classes (g-Bregman divergence and skew g-Jensen divergence) satisfy some properties similar to the Bregman or skew Jensen divergence. We show these g-divergences include divergences which belong to a class of f-divergence (the Hellinger distance, the chi-square divergence and the alpha-divergence in addition to the Kullback-Leibler divergence). Moreover, we derive an inequality between the g-Bregman divergence and the skew g-Jensen divergence and show this inequality is a generalization of Lin's inequality.

math.ST↗

arXiv2018

Divergence functions in dually flat spaces and their properties

In the field of statistics, many kind of divergence functions have been studied as an amount which measures the discrepancy between two probability distributions. In the differential geometrical approach in statistics (information geometry), dually flat spaces play a key role. In a dually flat space, there exist dual affine coordinate systems and strictly convex functions called potential and a canonical divergence is naturally introduced as a function of the affine coordinates and potentials. The canonical divergence satisfies a relational expression called triangular relation. This can be regarded as a generalization of the law of cosines in Euclidean space. In this paper, we newly introduce two kinds of divergences. The first divergence is a function of affine coordinates and it is consistent with the Jeffreys divergence for exponential or mixture families. For this divergence, we show that more relational equations and theorems similar to Euclidean space hold in addition to the law of cosines. The second divergences are functions of potentials and they are consistent with the Bhattacharyya distance for exponential families and are consistent with the Jensen-Shannon divergence for mixture families respectively. We derive an inequality between the the first and the second divergences and show that the inequality is a generalization of Lin's inequality.

stat.ME↗

arXiv2018

Improved Chebyshev inequality: new probability bounds with known supremum of PDF

In this paper, we derive new probability bounds for Chebyshev's inequality if the supremum of the probability density function is known. This result holds for one-dimensional or multivariate continuous probability distributions with finite mean and variance (covariance matrix). We also show that the similar result holds for specific discrete probability distributions.

stat.ME↗