arXiv subjects
Publications and source records attributed to Ambedkar Dukkipati.
In this paper we explore the possibility of using computational algebraic methods to analyze a class of consensus protocols. We state some necessary conditions for convergence under consensus protocols that are polynomials.
Low density parity-check (LDPC) codes are a class of linear block codes that are decoded by running belief propagation (BP) algorithm or log-likelihood ratio belief propagation (LLR-BP) over the factor graph of the code. One of the disadvantages of LDPC codes is the onset of an error floor at high values of signal to noise ratio caused by trapping sets. In this paper, we propose a two stage decoder to deal with different types of trapping sets. Oscillating trapping sets are taken care by the first stage of the decoder and the elementary trapping sets are handled by the second stage of the decoder. Simulation results on regular PEG (504,252,3,6) code shows that the proposed two stage decoder performs significantly better than the standard decoder.
The Gröbner basis detection (GBD) is defined as follows: Given a set of polynomials, decide whether there exists -and if "yes" find- a term order such that the set of polynomials is a Gröbner basis. This problem was shown to be NP-hard by Sturmfels and Wiegelmann. We show that GBD when studied in the context of zero dimensional ideals is also NP-hard. An algorithm to solve GBD for zero dimensional ideals is also proposed which runs in polynomial time if the number of indeterminates is a constant.
Border basis detection (BBD) is described as follows: given a set of generators of an ideal, decide whether that set of generators is a border basis of the ideal with respect to some order ideal. The motivation for this problem comes from a similar problem related to Gröbner bases termed as Gröbner basis detection (GBD) which was proposed by Gritzmann and Sturmfels (1993). GBD was shown to be NP-hard by Sturmfels and Wiegelmann (1996). In this paper, we investigate the computational complexity of BBD and show that it is NP-complete.
We show that various formulations (e.g., dual and Kullback-Csiszar iterations) of estimation of maximum entropy (ME) models can be transformed to solving systems of polynomial equations in several variables for which one can use celebrated Grobner bases methods. Posing of ME estimation as solving polynomial equations is possible, in the cases where feature functions (sufficient statistic) that provides the information about the underlying random variable in the form of expectations are integer valued.
Kullback-Leibler relative-entropy, in cases involving distributions resulting from relative-entropy minimization, has a celebrated property reminiscent of squared Euclidean distance: it satisfies an analogue of the Pythagoras' theorem. And hence, this property is referred to as Pythagoras' theorem of relative-entropy minimization or triangle equality and plays a fundamental role in geometrical approaches of statistical estimation theory like information geometry. Equvalent of Pythagoras' theorem in the generalized nonextensive formalism is established in (Dukkipati at el., Physica A, 361 (2006) 124-138). In this paper we give a detailed account of it.
Algebraic statistics is a recently evolving field, where one would treat statistical models as algebraic objects and thereby use tools from computational commutative algebra and algebraic geometry in the analysis and computation of statistical models. In this approach, calculation of parameters of statistical models amounts to solving set of polynomial equations in several variables, for which one can use celebrated Grobner bases theory. Owing to the important role of information theory in statistics, this paper as a first step, explores the possibility of describing maximum and minimum entropy (ME) models in the framework of algebraic statistics. We show that ME-models are toric models (a class of algebraic statistical models) when the constraint functions (that provide the information about the underlying random variable) are integer valued functions, and the set of statistical models that results from ME-methods are indeed an affine variety.
Though Shannon entropy of a probability measure $P$, defined as $- \int_{X} \frac{\ud P}{\ud μ} \ln \frac{\ud P}{\udμ} \ud μ$ on a measure space $(X, \mathfrak{M},μ)$, does not qualify itself as an information measure (it is not a natural extension of the discrete case), maximum entropy (ME) prescriptions in the measure-theoretic case are consistent with that of discrete case. In this paper, we study the measure-theoretic definitions of generalized information measures and discuss the ME prescriptions. We present two results in this regard: (i) we prove that, as in the case of classical relative-entropy, the measure-theoretic definitions of generalized relative-entropies, Rényi and Tsallis, are natural extensions of their respective discrete cases, (ii) we show that, ME prescriptions of measure-theoretic Tsallis entropy are consistent with the discrete case.
The measure-theoretic definition of Kullback-Leibler relative-entropy (KL-entropy) plays a basic role in the definitions of classical information measures. Entropy, mutual information and conditional forms of entropy can be expressed in terms of KL-entropy and hence properties of their measure-theoretic analogs will follow from those of measure-theoretic KL-entropy. These measure-theoretic definitions are key to extending the ergodic theorems of information theory to non-discrete cases. A fundamental theorem in this respect is the Gelfand-Yaglom-Perez (GYP) Theorem (Pinsker, 1960, Theorem. 2.4.2) which states that measure-theoretic relative-entropy equals the supremum of relative-entropies over all measurable partitions. This paper states and proves the GYP-theorem for Renyi relative-entropy of order greater than one. Consequently, the result can be easily extended to Tsallis relative-entropy.
By replacing linear averaging in Shannon entropy with Kolmogorov-Nagumo average (KN-averages) or quasilinear mean and further imposing the additivity constraint, Rényi proposed the first formal generalization of Shannon entropy. Using this recipe of Rényi, one can prepare only two information measures: Shannon and Rényi entropy. Indeed, using this formalism Rényi characterized these additive entropies in terms of axioms of quasilinear mean. As additivity is a characteristic property of Shannon entropy, pseudo-additivity of the form $x \oplus_{q} y = x + y + (1-q)x y$ is a characteristic property of nonextensive (or Tsallis) entropy. One can apply Rényi's recipe in the nonextensive case by replacing the linear averaging in Tsallis entropy with KN-averages and thereby imposing the constraint of pseudo-additivity. In this paper we show that nonextensive entropy is unique under the Rényi's recipe, and there by give a characterization.
As additivity is a characteristic property of the classical information measure, Shannon entropy, pseudo-additivity is a characteristic property of Tsallis entropy. Renyi generalized Shannon entropy by means of Kolmogorov-Nagumo averages, by imposing additivity as a constraint.In this paper we show that there exists no generalization for Tsallis entropy, by means of Kolmogorov-Nagumo averages, which preserves the pseudo-additivity.
Kullback-Leibler relative-entropy has unique properties in cases involving distributions resulting from relative-entropy minimization. Tsallis relative-entropy is a one parameter generalization of Kullback-Leibler relative-entropy in the nonextensive thermostatistics. In this paper, we present the properties of Tsallis relative-entropy minimization and present some differences with the classical case. In the representation of such a minimum relative-entropy distribution, we highlight the use of the q-product, an operator that has been recently introduced to derive the mathematical structure behind the Tsallis statistics. One of our main results is generalization of triangle equality of relative-entropy minimization to the nonextensive case.
Boltzmann selection is an important selection mechanism in evolutionary algorithms as it has theoretical properties which help in theoretical analysis. However, Boltzmann selection is not used in practice because a good annealing schedule for the `inverse temperature' parameter is lacking. In this paper we propose a Cauchy annealing schedule for Boltzmann selection scheme based on a hypothesis that selection-strength should increase as evolutionary process goes on and distance between two selection strengths should decrease for the process to converge. To formalize these aspects, we develop formalism for selection mechanisms using fitness distributions and give an appropriate measure for selection-strength. In this paper, we prove an important result, by which we derive an annealing schedule called Cauchy annealing schedule. We demonstrate the novelty of proposed annealing schedule using simulations in the framework of genetic algorithms.
Generalized evolutionary algorithm based on Tsallis canonical distribution is proposed. The algorithm uses Tsallis generalized canonical distribution to weigh the configurations for `selection' instead of Gibbs-Boltzmann distribution. Our simulation results show that for an appropriate choice of non-extensive index that is offered by Tsallis statistics, evolutionary algorithms based on this generalization outperform algorithms based on Gibbs-Boltzmann distribution.