arXiv subjects
Publications and source records attributed to Dmitry Panchenko.
We show that, under the conditions known to imply the validity of the Parisi formula, if the generic Sherrington-Kirkpatrick Hamiltonian contains a $p$-spin term then the Ghirlanda-Guerra identities for the $p$th power of the overlap hold in a strong sense without averaging. This implies strong version of the extended Ghirlanda-Guerra identities for mixed $p$-spin models than contain terms for all even $p\geq 2$ and $p=1.$
We consider a symmetric positive definite weakly exchangeable infinite random matrix and show that, under the technical condition that its elements take a finite number of values, the Ghirlanda--Guerra identities imply ultrametricity.
We present a detailed proof of the Dovbysh-Sudakov representation for symmetric positive definite weakly exchangeable infinite random arrays, called Gram-de Finetti matrices, which is based on the representation result of Aldous and Hoover for arbitrary (not necessarily positive definite) symmetric weakly exchangeable arrays.
A clear understanding the behavior of the error probability (EP) as a function of signal-to-noise ratio (SNR) and other system parameters is fundamental for assessing the design of digital wireless communication systems.We propose an analytical framework based on the log-concavity property of the EP which we prove for a wide family of multidimensional modulation formats in the presence of Gaussian disturbances and fading. Based on this property, we construct a class of local bounds for the EP that improve known generic bounds in a given region of the SNR and are invertible, as well as easily tractable for further analysis. This concept is motivated by the fact that communication systems often operate with performance in a certain region of interest (ROI) and, thus, it may be advantageous to have tighter bounds within this region instead of generic bounds valid for all SNRs. We present a possible application of these local bounds, but their relevance is beyond the example made in this paper.
It was proved by Michel Talagrand in [10] that the Parisi formula for the free energy in the Sherrington-Kirkpatrick model is differentiable with respect to inverse temperature parameter. We present a simpler proof of this result by using approximate solutions in the Parisi formula and give one example of application of the differentiability to prove non self-averaging of the overlap outside of the replica symmetric region.
In order to study certain questions concerning the distribution of the overlap in Sherrington--Kirkpatrick type models, such as the chaos and ultrametricity problems, it seems natural to study the free energy of multiple systems with constrained overlaps. One can write analogues of Guerra's replica symmetry breaking bound for such systems but it is not at all obvious how to choose informative functional order parameters in these bounds. We were able to make some progress for spherical pure $p$-spin SK models where many computations can be made explicitly. For pure 2-spin model we prove ultrametricity and chaos in an external field. For the pure $p$-spin model for even $p>4$ without an external field we describe two possible values of the overlap of two systems at different temperatures. We also prove a somewhat unexpected result which shows that in the 2-spin model the support of the joint overlap distribution is not always witnessed at the level of the free energy and, for example, ultrametricity holds only in a weak sense.
New results about Poisson-Dirichlet point processes and Derrida-Ruelle cascades allow us to express Guerra's interpolation entirely in the language of Derrida-Ruelle cascades and to streamline Guerra's computations. Moreover, our approach clarifies the nature of the error terms along the interpolation.
Talagrand's positivity principle states that one can slightly perturb a Hamiltonian in the Sherrington-Kirkpatrick model in such a way that the overlap of two configurations under the perturbed Gibbs' measure will become typically nonnegative. In this note we observe that abstracting from the setting of the SK model only improves the result and does not require any modifications in Talagrand's argument. In this version, for example, positivity principle immediately applies to the setting of Aizenman-Sims-Starr interpolation. Also, abstracting from the SK model improves the conditions in the Ghirlanda-Guerra identities and as a consequence results in a perturbation of smaller order necessary to ensure positivity of the overlap.
We develop a cavity method in the spherical Sherrington-Kirkpatrick model at high temperature and small external field. As one application we compute the limit of the covariance matrix for fluctuations of the overlap and magnetization.
Recently, Michel Talagrand computed the large deviations limit $\lim_{N\to\infty}(Na)^{-1}\log \e Z_N^a$ for the moments of the partition function $Z_N$ in the Sherrington-Kirkpatrick model for all real $a\geq 0.$ For $a\geq 1$ the limit is given by Guerra's inverse bound and this result extends the classical physicist's replica method that corresponds to integer $a.$ We give a new proof for $a\geq 1$ in the case of the pure $p$-spin SK model that provides a strong exponential control of the overlap.
We introduce and study several measures of complexity of functions from the convex hull of a given base class. These complexity measures take into account the sparsity of the weights of a convex combination as well as certain clustering properties of the base functions involved in it. We prove new upper confidence bounds on the generalization error of ensemble (voting) classification algorithms that utilize the new complexity measures along with the empirical distributions of classification margins, providing a better explanation of generalization performance of large margin classification methods.
Recently Michel Talagrand gave a rigorous proof of the Parisi formula in the Sherrington-Kirkpatrick model. In this paper we build upon the methodology developed by Talagrand and extend his result to the class of SK type models in which the spins have arbitrary prior distribution on a bounded subset of the real line.
We conjecture that the Parisi functional in the SK model is convex in the functional order parameter and prove a partial result that shows the convexity along one-sided directions. A consequence of this result is log-convexity of L_1 norm for a class or random variables.
This paper has been withdrawn since a trivial proof of the result has been pointed out to the author.
We investigate measures of complexity of function classes based on continuity moduli of Gaussian and Rademacher processes. For Gaussian processes, we obtain bounds on the continuity modulus on the convex hull of a function class in terms of the same quantity for the class itself. We also obtain new bounds on generalization error in terms of localized Rademacher complexities. This allows us to prove new results about generalization performance for convex hulls in terms of characteristics of the base class. As a byproduct, we obtain a simple proof of some of the known bounds on the entropy of convex hulls.
The inequality of Vapnik and Chervonenkis controls the expectation of the function by its sample average uniformly over a VC-major class of functions taking into account the size of the expectation. Using Talagrand's kernel method we prove a similar result for the classes of functions for which Dudley's uniform entropy integral or bracketing entropy integral is finite.
We prove new probabilistic upper bounds on generalization error of complex classifiers that are combinations of simple classifiers. Such combinations could be implemented by neural networks or by voting methods of combining the classifiers, such as boosting and bagging. The bounds are in terms of the empirical distribution of the margin of the combined classifier. They are based on the methods of the theory of Gaussian and empirical processes (comparison inequalities, symmetrization method, concentration inequalities) and they improve previous results of Bartlett (1998) on bounding the generalization error of neural networks in terms of l_1-norms of the weights of neurons and of Schapire, Freund, Bartlett and Lee (1998) on bounding the generalization error of boosting. We also obtain rates of convergence in Levy distance of empirical margin distribution to the true margin distribution uniformly over the classes of classifiers and prove the optimality of these rates.
A problem of bounding the generalization error of a classifier f in H, where H is a "base" class of functions (classifiers), is considered. This problem frequently occurs in computer learning, where efficient algorithms of combining simple classifiers into a complex one (such as boosting and bagging) have attracted a lot of attention. Using Talagrand's concentration inequalities for empirical processes, we obtain new sharper bounds on the generalization error of combined classifiers that take into account both the empirical distribution of "classification margins'' and an "approximate dimension" of the classifiers and study the performance of these bounds in several experiments with learning algorithms.