Searcharxiv⌕ Search

arXiv subjects

Guanyang Wang

Publications and source records attributed to Guanyang Wang.

At least 37 records · Page 2Linked to original sources
arXiv2024

A phase transition in sampling from Restricted Boltzmann Machines

Restricted Boltzmann Machines are a class of undirected graphical models that play a key role in deep learning and unsupervised learning. In this study, we prove a phase transition phenomenon in the mixing time of the Gibbs sampler for a one-parameter Restricted Boltzmann Machine. Specifically, the mixing time varies logarithmically, polynomially, and exponentially with the number of vertices depending on whether the parameter $c$ is above, equal to, or below a critical value $c_\star\approx-5.87$. A key insight from our analysis is the link between the Gibbs sampler and a dynamical system, which we utilize to quantify the former based on the behavior of the latter. To study the critical case $c= c_\star$, we develop a new isoperimetric inequality for the sampler's stationary distribution by showing that the distribution is nearly log-concave.

cs.LG↗
arXiv2024

Putting all eggs in one basket: some insights from a correlation inequality

We give examples of situations -- stochastic production, military tactics, corporate merger -- where it is beneficial to concentrate risk rather than to diversify it, that is, to put all eggs in one basket. Our examples admit a dual interpretation: as optimal strategies of a single player (the `principal') or, alternatively, as dominant strategies in a non-cooperative game with multiple players (the `agents'). The key mathematical result can be formulated in terms of a convolution structure on the set of increasing functions on a Boolean lattice (the lattice of subsets of a finite set). This generalizes the well-known Harris inequality from statistical physics and discrete mathematics; we give a simple self-contained proof of this result, and prove a further generalization based on the game-theoretic approach.

math.PR↗
arXiv2024

When are Unbiased Monte Carlo Estimators More Preferable than Biased Ones?

Due to the potential benefits of parallelization, designing unbiased Monte Carlo estimators, primarily in the setting of randomized multilevel Monte Carlo, has recently become very popular in operations research and computational statistics. However, existing work primarily substantiates the benefits of unbiased estimators at an intuitive level or using empirical evaluations. The intuition being that unbiased estimators can be replicated in parallel enabling fast estimation in terms of wall-clock time. This intuition ignores that, typically, bias will be introduced due to impatience because most unbiased estimators necesitate random completion times. This paper provides a mathematical framework for comparing these methods under various metrics, such as completion time and overall computational cost. Under practical assumptions, our findings reveal that unbiased methods typically have superior completion times - the degree of superiority being quantifiable through the tail behavior of their running time distribution - but they may not automatically provide substantial savings in overall computational costs. We apply our findings to Markov Chain Monte Carlo and Multilevel Monte Carlo methods to identify the conditions and scenarios where unbiased methods have an advantage, thus assisting practitioners in making informed choices between unbiased and biased methods.

stat.CO↗
arXiv2024

Quadratic Speed-up in Infinite Variance Quantum Monte Carlo

In this study, we give an extension of Montanaro's arXiv/archive:1504.06987 quantum Monte Carlo method, tailored for computing expected values of random variables that exhibit infinite variance. This addresses a challenge in analyzing heavy-tailed distributions, which are commonly encountered in various scientific and engineering fields. Our quantum algorithm efficiently estimates means for variables with a finite $(1+δ)^{\text{th}}$ moment, where $δ$ lies between 0 and 1. It provides a quadratic speedup over the classical Monte Carlo method in both the accuracy parameter $ε$ and the specified moment of the distribution. We establish both classical and quantum lower bounds, showcasing the near-optimal efficiency of our algorithm among quantum methods. Our work focuses not on creating new algorithms, but on analyzing the execution of existing algorithms with available additional information about the random variable. Additionally, we categorize these scenarios and demonstrate a hierarchy in the types of supplementary information that can be provided.

quant-ph↗
arXiv2023

Pattern detection in bipartite networks: a review of terminology, applications and methods

Two dimensional matrices with binary (0/1) entries are a common data structure in many research fields. Examples include ecology, economics, mathematics, physics, psychometrics and others. Because the columns and rows of these matrices represent distinct entities, they can equivalently be expressed as a pair of bipartite networks that are linked by projection. A variety of diversity statistics and network metrics can then be used to quantify patterns in these matrices and networks. But what should these patterns be compared to? In all of these disciplines, researchers have recognized the necessity of comparing an empirical matrix to a benchmark set of "null" matrices created by randomizing certain elements of the original data. This common need has nevertheless promoted the independent development of methodologies by researchers who come from different backgrounds and use different terminology. Here, we provide a multidisciplinary review of randomization techniques for matrices representing binary, bipartite networks. We aim to translate the concepts from different technical domains into a common language that is accessible to a broad scientific audience. Specifically, after briefly reviewing examples of binary matrix structures across different fields, we introduce the major approaches and common strategies for randomizing these matrices. We then explore the details of and performance of specific techniques, and discuss their limitations and computational challenges. In particular, we focus on the conceptual importance and implementation of structural constraints on the randomization, such as preserving row or columns sums of the original matrix in each of the randomized matrices. Our review serves both as a guide for empiricists in different disciplines, as well as a reference point for researchers working on theoretical and methodological developments in matrix randomization methods.

physics.data-an↗
arXiv2023

Optimal randomized multilevel Monte Carlo for repeatedly nested expectations

The estimation of repeatedly nested expectations is a challenging task that arises in many real-world systems. However, existing methods generally suffer from high computational costs when the number of nestings becomes large. Fix any non-negative integer $D$ for the total number of nestings. Standard Monte Carlo methods typically cost at least $\mathcal{O}(\varepsilon^{-(2+D)})$ and sometimes $\mathcal{O}(\varepsilon^{-2(1+D)})$ to obtain an estimator up to $\varepsilon$-error. More advanced methods, such as multilevel Monte Carlo, currently only exist for $D = 1$. In this paper, we propose a novel Monte Carlo estimator called $\mathsf{READ}$, which stands for "Recursive Estimator for Arbitrary Depth.'' Our estimator has an optimal computational cost of $\mathcal{O}(\varepsilon^{-2})$ for every fixed $D$ under suitable assumptions, and a nearly optimal computational cost of $\mathcal{O}(\varepsilon^{-2(1 + δ)})$ for any $0 < δ< \frac12$ under much more general assumptions. Our estimator is also unbiased, which makes it easy to parallelize. The key ingredients in our construction are an observation of the problem's recursive structure and the recursive use of the randomized multilevel Monte Carlo method.

stat.CO↗
arXiv2023

Metropolis-Hastings transition kernel couplings

Couplings play a central role in the analysis of Markov chain convergence and in the construction of novel Markov chain Monte Carlo estimators, diagnostics, and variance reduction techniques. The set of possible couplings is often intractable, frustrating the search for tight bounds and efficient estimators. To address this challenge for algorithms in the Metropolis-Hastings (MH) family, we establish a simple characterization of the set of MH transition kernel couplings. We then extend this result to describe the set of maximal couplings of the MH kernel, resolving an open question of O'Leary et al.. Our results represent an advance in understanding the MH transition kernel and a step forward for coupling this popular class of algorithms.

math.ST↗
arXiv2022

Unbiased Optimal Stopping via the MUSE

We propose a new unbiased estimator for estimating the utility of the optimal stopping problem. The MUSE, short for Multilevel Unbiased Stopping Estimator, constructs the unbiased Multilevel Monte Carlo (MLMC) estimator at every stage of the optimal stopping problem in a backward recursive way. In contrast to traditional sequential methods, the MUSE can be implemented in parallel. We prove the MUSE has finite variance, finite computational complexity, and achieves $ε$-accuracy with $O(1/ε^2)$ computational cost under mild conditions. We demonstrate MUSE empirically in an option pricing problem involving a high-dimensional input and the use of many parallel processors.

stat.CO↗
arXiv2022

Unbiased Multilevel Monte Carlo methods for intractable distributions: MLMC meets MCMC

Constructing unbiased estimators from Markov chain Monte Carlo (MCMC) outputs is a difficult problem that has recently received a lot of attention in the statistics and machine learning communities. However, the current unbiased MCMC framework only works when the quantity of interest is an expectation, which excludes many practical applications. In this paper, we propose a general method for constructing unbiased estimators for functions of expectations and extend it to construct unbiased estimators for nested expectations. Our approach combines and generalizes the unbiased MCMC and Multilevel Monte Carlo (MLMC) methods. In contrast to traditional sequential methods, our estimator can be implemented on parallel processors. We show that our estimator has a finite variance and computational complexity and can achieve $\varepsilon$-accuracy within the optimal $O(1/\varepsilon^2)$ computational cost under mild conditions. Our numerical experiments confirm our theoretical findings and demonstrate the benefits of unbiased estimators in the massively parallel regime.

stat.CO↗
arXiv2022

Spectral Telescope: Convergence Rate Bounds for Random-Scan Gibbs Samplers Based on a Hierarchical Structure

Random-scan Gibbs samplers possess a natural hierarchical structure. The structure connects Gibbs samplers targeting higher dimensional distributions to those targeting lower dimensional ones. This leads to a quasi-telescoping property of their spectral gaps. Based on this property, we derive three new bounds on the spectral gaps and convergence rates of Gibbs samplers on general domains. The three bounds relate a chain's spectral gap to, respectively, the correlation structure of the target distribution, a class of random walk chains, and a collection of influence matrices. Notably, one of our results generalizes the technique of spectral independence, which has received considerable attention for its success on finite domains, to general state spaces. We illustrate our methods through a sampler targeting the uniform distribution on a corner of an $n$-cube.

math.PR↗
arXiv2022

Repeated Averages on Graphs

Sourav Chatterjee, Persi Diaconis, Allan Sly and Lingfu Zhang, prompted by a question of Ramis Movassagh, renewed the study of a process proposed in the early 1980s by Jean Bourgain. A state vector $v \in \mathbb R^n$, labeled with the vertices of a connected graph, $G$, changes in discrete time steps following the simple rule that at each step a random edge $(i,j)$ is picked and $v_i$ and $v_j$ are both replaced by their average $(v_i+v_j)/2$. It is easy to see that the value associated with each vertex converges to $1/n$. The question was how quickly will $v$ be $ε$-close to uniform in the $L^{1}$ norm in the case of the complete graph, $K_{n}$, when $v$ is initialized as a standard basis vector that takes the value 1 on one coordinate, and zeros everywhere else. They have established a sharp cutoff of $\frac{1}{2\log 2}n\log n + O(n\sqrt{\log n})$. Our main result is to prove, that $\frac{(1-ε)}{2\log2}n\log n-O(n)$ is a general lower bound for all connected graphs on $n$ nodes. We also get sharp magnitude of $t_{ε,1}$ for several important families of graphs, including star, expander, dumbbell, and cycle. In order to establish our results we make several observations about the process, such as the worst case initialization is always a standard basis vector. Our results add to the body of work of Aldous, Aldous and Lanoue, Quattropani and Sau, Cao, Olshevsky and Tsitsiklis, and others. The renewed interest is due to an analogy to a question related to the Google's supremacy circuit. For the proof of our main theorem we employ a concept that we call 'augmented entropy function' which may find independent interest in the computer science and probability theory communities.

math.PR↗
arXiv2022

On the Minimax Spherical Designs

Distributing points on a (possibly high-dimensional) sphere with minimal energy is a long-standing problem in and outside the field of mathematics. This paper considers a novel energy function that arises naturally from statistics and combinatorial optimization, and studies its theoretical properties. Our result solves both the exact optimal spherical point configurations in certain cases and the minimal energy asymptotics under general assumptions. Connections between our results and the L1-Principal Component analysis and Quasi-Monte Carlo methods are also discussed.

math.CO↗
arXiv2021

On the Theoretical Properties of the Exchange Algorithm

The exchange algorithm is one of the most popular extensions of the Metropolis--Hastings algorithm to sample from doubly-intractable distributions. However, the theoretical exploration of the exchange algorithm is very limited. For example, natural questions like `Does exchange algorithm converge at a geometric rate?' or `Does the exchange algorithm admit a Central Limit Theorem?' have not been answered yet. In this paper, we study the theoretical properties of the exchange algorithm, in terms of asymptotic variance and convergence speed. We compare the exchange algorithm with the original Metropolis--Hastings algorithm and provide both necessary and sufficient conditions for the geometric ergodicity of the exchange algorithm. Moreover, we prove that our results can be applied to various practical applications such as location models, Gaussian models, Poisson models, and a large class of exponential families, which includes most of the practical applications of the exchange algorithm. A central limit theorem for the exchange algorithm is also established. Our results justify the theoretical usefulness of the exchange algorithm.

stat.CO↗
arXiv2021

Exact Convergence Rate Analysis of the Independent Metropolis-Hastings Algorithms

A well-known difficult problem regarding Metropolis-Hastings algorithms is to get sharp bounds on their convergence rates. Moreover, a fundamental but often overlooked problem in Markov chain theory is to study the convergence rates for different initializations. In this paper, we study the two issues mentioned above of the Independent Metropolis-Hastings (IMH) algorithms on both general and discrete state spaces. We derive the exact convergence rate and prove that the IMH algorithm's different deterministic initializations have the same convergence rate. We get the exact convergence speed for IMH algorithms on general state spaces.

math.PR↗
arXiv2020

Maximal couplings of the Metropolis-Hastings algorithm

Couplings play a central role in the analysis of Markov chain Monte Carlo algorithms and appear increasingly often in the algorithms themselves, e.g. in convergence diagnostics, parallelization, and variance reduction techniques. Existing couplings of the Metropolis-Hastings algorithm handle the proposal and acceptance steps separately and fall short of the upper bound on one-step meeting probabilities given by the coupling inequality. This paper introduces maximal couplings which achieve this bound while retaining the practical advantages of current methods. We consider the properties of these couplings and examine their behavior on a selection of numerical examples.

stat.CO↗
arXiv2020

Integrated Cooling (i-Cool) Textile of Heat Conduction and Sweat Transportation for Personal Perspiration Management

Perspiration evaporation plays an indispensable role in human body heat dissipation. However, conventional textiles show limited perspiration management capability in moderate/profuse perspiration scenarios, i.e. low evaporation ability, ineffective evaporative cooling effect, and resultant human body dehydration and electrolyte disorder. Here, we propose a novel concept of integrated cooling (i-Cool) textile of heat conduction and sweat transportation for personal perspiration management based on unique functional structure design. By integrating heat conductive pathways and water transport channels decently, this textile not only shows the capability of liquid water wicking, but also exhibits superior evaporation rate than traditional textiles. Furthermore, compared with cotton, about 2.8 $^\circ$C cooling effect causing less than one third amount of dehydration has also been demonstrated on the artificial sweating skin platform with feedback control loop simulating human body perspiration situation. Moreover, the practical application feasibility of the i-Cool textile design principles has been validated as well. Owing to its exceptional personal perspiration management performance in liquid water wicking, fast evaporation, efficient cooling effect and reduced human body dehydration/electrolyte loss, we expect this i-Cool textile provides promising design guidelines for next-generation personal perspiration management textile.

physics.app-ph↗
arXiv2019

A Fast MCMC for the Uniform Sampling of Binary Matrices with Fixed Margins

Uniform sampling of binary matrix with fixed margins is an important and difficult problem in statistics, computer science, ecology and so on. The well-known swap algorithm would be inefficient when the size of the matrix becomes large or when the matrix is too sparse/dense. Here we propose the Rectangle Loop algorithm, a Markov chain Monte Carlo algorithm to sample binary matrices with fixed margins uniformly. Theoretically the Rectangle Loop algorithm is better than the swap algorithm in Peskun's order. Empirically studies also demonstrates the Rectangle Loop algorithm is remarkablely more efficient than the swap algorithm.

stat.CO↗