arXiv subjects

Chihao Zhang

Publications and source records attributed to Chihao Zhang.

35 records · Page 2Linked to original sources

arXiv2022

Information-theoretic Classification Accuracy: A Criterion that Guides Data-driven Combination of Ambiguous Outcome Labels in Multi-class Classification

Outcome labeling ambiguity and subjectivity are ubiquitous in real-world datasets. While practitioners commonly combine ambiguous outcome labels for all data points (instances) in an ad hoc way to improve the accuracy of multi-class classification, there lacks a principled approach to guide the label combination for all data points by any optimality criterion. To address this problem, we propose the information-theoretic classification accuracy (ITCA), a criterion that balances the trade-off between prediction accuracy (how well do predicted labels agree with actual labels) and classification resolution (how many labels are predictable), to guide practitioners on how to combine ambiguous outcome labels. To find the optimal label combination indicated by ITCA, we propose two search strategies: greedy search and breadth-first search. Notably, ITCA and the two search strategies are adaptive to all machine-learning classification algorithms. Coupled with a classification algorithm and a search strategy, ITCA has two uses: improving prediction accuracy and identifying ambiguous labels. We first verify that ITCA achieves high accuracy with both search strategies in finding the correct label combinations on synthetic and real data. Then we demonstrate the effectiveness of ITCA in diverse applications including medical prognosis, cancer survival prediction, user demographics prediction, and cell type classification. We also provide theoretical insights into ITCA by studying the oracle and the linear discriminant analysis classification algorithms. Python package itca (available at https://github.com/JSB-UCLA/ITCA) implements ITCA and search strategies.

stat.ML↗

arXiv2021

Understanding Bandits with Graph Feedback

The bandit problem with graph feedback, proposed in [Mannor and Shamir, NeurIPS 2011], is modeled by a directed graph $G=(V,E)$ where $V$ is the collection of bandit arms, and once an arm is triggered, all its incident arms are observed. A fundamental question is how the structure of the graph affects the min-max regret. We propose the notions of the fractional weak domination number $δ^*$ and the $k$-packing independence number capturing upper bound and lower bound for the regret respectively. We show that the two notions are inherently connected via aligning them with the linear program of the weakly dominating set and its dual -- the fractional vertex packing set respectively. Based on this connection, we utilize the strong duality theorem to prove a general regret upper bound $O\left(\left( δ^*\log |V|\right)^{\frac{1}{3}}T^{\frac{2}{3}}\right)$ and a lower bound $Ω\left(\left(δ^*/α\right)^{\frac{1}{3}}T^{\frac{2}{3}}\right)$ where $α$ is the integrality gap of the dual linear program. Therefore, our bounds are tight up to a $\left(\log |V|\right)^{\frac{1}{3}}$ factor on graphs with bounded integrality gap for the vertex packing problem including trees and graphs with bounded degree. Moreover, we show that for several special families of graphs, we can get rid of the $\left(\log |V|\right)^{\frac{1}{3}}$ factor and establish optimal regret.

cs.LG↗

arXiv2021

Matrix Normal PCA for Interpretable Dimension Reduction and Graphical Noise Modeling

Principal component analysis (PCA) is one of the most widely used dimension reduction and multivariate statistical techniques. From a probabilistic perspective, PCA seeks a low-dimensional representation of data in the presence of independent identical Gaussian noise. Probabilistic PCA (PPCA) and its variants have been extensively studied for decades. Most of them assume the underlying noise follows a certain independent identical distribution. However, the noise in the real world is usually complicated and structured. To address this challenge, some variants of PCA for data with non-IID noise have been proposed. However, most of the existing methods only assume that the noise is correlated in the feature space while there may exist two-way structured noise. To this end, we propose a powerful and intuitive PCA method (MN-PCA) through modeling the graphical noise by the matrix normal distribution, which enables us to explore the structure of noise in both the feature space and the sample space. MN-PCA obtains a low-rank representation of data and the structure of noise simultaneously. And it can be explained as approximating data over the generalized Mahalanobis distance. We develop two algorithms to solve this model: one maximizes the regularized likelihood, the other exploits the Wasserstein distance, which is more robust. Extensive experiments on various data demonstrate their effectiveness.

cs.LG↗

arXiv2020

Rapid mixing from spectral independence beyond the Boolean domain

We extend the notion of spectral independence (introduced by Anari, Liu, and Oveis Gharan [ALO20]) from the Boolean domain to general discrete domains. This property characterises distributions with limited correlations, and implies that the corresponding Glauber dynamics is rapidly mixing. As a concrete application, we show that Glauber dynamics for sampling proper $q$-colourings mixes in polynomial-time for the family of triangle-free graphs with maximum degree $Δ$ provided $q\ge (α^*+δ)Δ$ where $α^*\approx 1.763$ is the unique solution to $α^*=\exp(1/α^*)$ and $δ>0$ is any constant. This is the first efficient algorithm for sampling proper $q$-colourings in this regime with possibly unbounded $Δ$. Our main tool of establishing spectral independence is the recursive coupling by Goldberg, Martin, and Paterson [GMP05].

cs.DS↗

arXiv2020

Distributed Bayesian Matrix Decomposition for Big Data Mining and Clustering

Matrix decomposition is one of the fundamental tools to discover knowledge from big data generated by modern applications. However, it is still inefficient or infeasible to process very big data using such a method in a single machine. Moreover, big data are often distributedly collected and stored on different machines. Thus, such data generally bear strong heterogeneous noise. It is essential and useful to develop distributed matrix decomposition for big data analytics. Such a method should scale up well, model the heterogeneous noise, and address the communication issue in a distributed system. To this end, we propose a distributed Bayesian matrix decomposition model (DBMD) for big data mining and clustering. Specifically, we adopt three strategies to implement the distributed computing including 1) the accelerated gradient descent, 2) the alternating direction method of multipliers (ADMM), and 3) the statistical inference. We investigate the theoretical convergence behaviors of these algorithms. To address the heterogeneity of the noise, we propose an optimal plug-in weighted average that reduces the variance of the estimation. Synthetic experiments validate our theoretical results, and real-world experiments show that our algorithms scale up well to big data and achieves superior or competing performance compared to other distributed methods.

cs.LG↗

arXiv2019

Fast sampling and counting k-SAT solutions in the local lemma regime

We give new algorithms based on Markov chains to sample and approximately count satisfying assignments to $k$-uniform CNF formulas where each variable appears at most $d$ times. For any $k$ and $d$ satisfying $kd<n^{o(1)}$ and $k\ge 20\log k + 20\log d + 60$, the new sampling algorithm runs in close to linear time, and the counting algorithm runs in close to quadratic time. Our approach is inspired by Moitra (JACM, 2019) which remarkably utilizes the Lovász local lemma in approximate counting. Our main technical contribution is to use the local lemma to bypass the connectivity barrier in traditional Markov chain approaches, which makes the well developed MCMC method applicable on disconnected state spaces such as SAT solutions. The benefit of our approach is to avoid the enumeration of local structures and obtain fixed polynomial running times, even if $k=ω(1)$ or $d=ω(1)$.

cs.DS↗

arXiv2019

Counting hypergraph colorings in the local lemma regime

We give a fully polynomial-time approximation scheme (FPTAS) to count the number of $q$-colorings for $k$-uniform hypergraphs with maximum degree $Δ$ if $k\ge 28$ and $q >357Δ^{\frac{14}{k-14}}$ . We also obtain a polynomial-time almost uniform sampler if $q>931Δ^{\frac{16}{k-16/3}}$. These are the first approximate counting and sampling algorithms in the regime $q\llΔ$ (for large $Δ$ and $k$) without any additional assumptions. Our method is based on the recent work of Moitra (STOC, 2017). One important contribution of ours is to remove the dependency of $k$ and $Δ$ in Moitra's approach.

cs.DS↗

arXiv2018

Zeros of Holant problems: locations and algorithms

We present fully polynomial-time (deterministic or randomised) approximation schemes for Holant problems, defined by a non-negative constraint function satisfying a generalised second order recurrence modulo a couple of exceptional cases. As a consequence, any non-negative Holant problem on cubic graphs has an efficient approximation algorithm unless the problem is equivalent to approximately counting perfect matchings, a central open problem in the area. This is in sharp contrast to the computational phase transition shown by 2-state spin systems on cubic graphs. Our main technique is the recently established connection between zeros of graph polynomials and approximate counting. We also use the "winding" technique to deduce the second result on cubic graphs.

cs.DS↗

arXiv2017

Bayesian Joint Matrix Decomposition for Data Integration with Heterogeneous Noise

Matrix decomposition is a popular and fundamental approach in machine learning and data mining. It has been successfully applied into various fields. Most matrix decomposition methods focus on decomposing a data matrix from one single source. However, it is common that data are from different sources with heterogeneous noise. A few of matrix decomposition methods have been extended for such multi-view data integration and pattern discovery. While only few methods were designed to consider the heterogeneity of noise in such multi-view data for data integration explicitly. To this end, we propose a joint matrix decomposition framework (BJMD), which models the heterogeneity of noise by Gaussian distribution in a Bayesian framework. We develop two algorithms to solve this model: one is a variational Bayesian inference algorithm, which makes full use of the posterior distribution; and another is a maximum a posterior algorithm, which is more scalable and can be easily paralleled. Extensive experiments on synthetic and real-world datasets demonstrate that BJMD considering the heterogeneity of noise is superior or competitive to the state-of-the-art methods.

cs.CV↗

arXiv2016

An FPTAS for Counting Proper Four-Colorings on Cubic Graphs

Graph coloring is arguably the most exhaustively studied problem in the area of approximate counting. It is conjectured that there is a fully polynomial-time (randomized) approximation scheme (FPTAS/FPRAS) for counting the number of proper colorings as long as $q \geq Δ+ 1$, where $q$ is the number of colors and $Δ$ is the maximum degree of the graph. The bound of $q = Δ+ 1$ is the uniqueness threshold for Gibbs measure on $Δ$-regular infinite trees. However, the conjecture remained open even for any fixed $Δ\geq 3$ (The cases of $Δ=1, 2$ are trivial). In this paper, we design an FPTAS for counting the number of proper $4$-colorings on graphs with maximum degree $3$ and thus confirm the conjecture in the case of $Δ=3$. This is the first time to achieve this optimal bound of $q = Δ+ 1$. Previously, the best FPRAS requires $q > \frac{11}{6} Δ$ and the best deterministic FPTAS requires $q > 2.581Δ+ 1$ for general graphs. In the case of $Δ=3$, the best previous result is an FPRAS for counting proper 5-colorings. We note that there is a barrier to go beyond $q = Δ+ 2$ for single-site Glauber dynamics based FPRAS and we overcome this by correlation decay approach. Moreover, we develop a number of new techniques for the correlation decay approach which can find applications in other approximate counting problems.

cs.DS↗

arXiv2015

Canonical Paths for MCMC: from Art to Science

Markov Chain Monte Carlo (MCMC) method is a widely used algorithm design scheme with many applications. To make efficient use of this method, the key step is to prove that the Markov chain is rapid mixing. Canonical paths is one of the two main tools to prove rapid mixing. However, there are much fewer success examples comparing to coupling, the other main tool. The main reason is that there is no systematic approach or general recipe to design canonical paths. Building up on a previous exploration by McQuillan, we develop a general theory to design canonical paths for MCMC: We reduce the task of designing canonical paths to solving a set of linear equations, which can be automatically done even by a machine. Making use of this general approach, we obtain fully polynomial-time randomized approximation schemes (FPRAS) for counting the number of $b$-matching with $b\leq 7$ and $b$-edge-cover with $b\leq 2$. They are natural generalizations of matchings and edge covers for graphs. No polynomial time approximation was previously known for these problems.

cs.DS↗

arXiv2015

FPTAS for Hardcore and Ising Models on Hypergraphs

Hardcore and Ising models are two most important families of two state spin systems in statistic physics. Partition function of spin systems is the center concept in statistic physics which connects microscopic particles and their interactions with their macroscopic and statistical properties of materials such as energy, entropy, ferromagnetism, etc. If each local interaction of the system involves only two particles, the system can be described by a graph. In this case, fully polynomial-time approximation scheme (FPTAS) for computing the partition function of both hardcore and anti-ferromagnetic Ising model was designed up to the uniqueness condition of the system. These result are the best possible since approximately computing the partition function beyond this threshold is NP-hard. In this paper, we generalize these results to general physics systems, where each local interaction may involves multiple particles. Such systems are described by hypergraphs. For hardcore model, we also provide FPTAS up to the uniqueness condition, and for anti-ferromagnetic Ising model, we obtain FPTAS where a slightly stronger condition holds.

cs.DS↗

arXiv2015

Sampling colorings almost uniformly in sparse random graphs

The problem of sampling proper $q$-colorings from uniform distribution has been extensively studied. Most of existing samplers require $q\ge αΔ+β$ for some constants $α$ and $β$, where $Δ$ is the maximum degree of the graph. The problem becomes more challenging when the underlying graph has unbounded degree since even the decision of $q$-colorability becomes nontrivial in this situation. The Erdős-Rényi random graph $\mathcal{G}(n,d/n)$ is a typical class of such graphs and has received a lot of recent attention. In this case, the performance of a sampler is usually measured by the relation between $q$ and the average degree $d$. We are interested in the fully polynomial-time almost uniform sampler (FPAUS) and the state-of-the-art with such sampler for proper $q$-coloring on $\mathcal{G}(n,d/n)$ requires that $q\ge 5.5d$. In this paper, we design an FPAUS for proper $q$-colorings on $\mathcal{G}(n,d/n)$ by requiring that $q\ge 3d+O(1)$, which improves the best bound for the problem so far. Our sampler is based on the spatial mixing property of $q$-coloring on random graphs. The core of the sampler is a deterministic algorithm to estimate the marginal probability on blocks, which is computed by a novel block version of recursion for $q$-coloring on unbounded degree graphs.

cs.DS↗

arXiv2015

Spatial mixing and approximate counting for Potts model on graphs with bounded average degree

We propose a notion of contraction function for a family of graphs and establish its connection to the strong spatial mixing for spin systems. More specifically, we show that for anti-ferromagnetic Potts model on families of graphs characterized by a specific contraction function, the model exhibits strong spatial mixing, and if further the graphs exhibit certain local sparsity which are very natural and easy to satisfy by typical sparse graphs, then we also have FPTAS for computing the partition function. This new characterization of strong spatial mixing of multi-spin system does not require maximum degree of the graphs to be bounded, but instead it relates the decay of correlation of the model to a notion of effective average degree measured by the contraction of a function on the family of graphs. It also generalizes other notion of effective average degree which may determine the strong spatial mixing, such as the connective constant, whose connection to strong spatial mixing is only known for very simple models and is not extendable to general spin systems. As direct consequences: (1) we obtain FPTAS for the partition function of $q$-state anti-ferromagnetic Potts model with activity $0\leβ<1$ on graphs of maximum degree bounded by $d$ when $q> 3(1-β)d+1$, improving the previous best bound $β> 3(1-β)d$ and asymptotically approaching the inapproximability threshold $q=(1-β)d$, and (2) we obtain an efficient sampler (in the same sense of fully polynomial-time almost uniform sampler, FPAUS) for the Potts model on Erdős-Rényi random graph $\mathcal{G}(n,d/n)$ with sufficiently large constant $d$, provided that $q> 3(1-β)d+4$. In particular when $β=0$, the sampler becomes an FPAUS for for proper $q$-coloring in $\mathcal{G}(n,d/n)$ with $q> 3d+4$, improving the current best bound $q> 5.5d$ for FPAUS for $q$-coloring in $\mathcal{G}(n,d/n)$.

cs.DS↗

arXiv2014

The Complexity of Ferromagnetic Two-spin Systems with External Fields

We study the approximability of computing the partition function for ferromagnetic two-state spin systems. The remarkable algorithm by Jerrum and Sinclair showed that there is a fully polynomial-time randomized approximation scheme (FPRAS) for the special ferromagnetic Ising model with any given uniform external field. Later, Goldberg and Jerrum proved that it is #BIS-hard for Ising model if we allow inconsistent external fields on different nodes. In contrast to these two results, we prove that for any ferromagnetic two-state spin systems except the Ising model, there exists a threshold for external fields beyond which the problem is #BIS-hard, even if the external field is uniform.

cs.CC↗

arXiv2014

FPTAS for Weighted Fibonacci Gates and Its Applications

Fibonacci gate problems have severed as computation primitives to solve other problems by holographic algorithm and play an important role in the dichotomy of exact counting for Holant and CSP frameworks. We generalize them to weighted cases and allow each vertex function to have different parameters, which is a much boarder family and #P-hard for exactly counting. We design a fully polynomial-time approximation scheme (FPTAS) for this generalization by correlation decay technique. This is the first deterministic FPTAS for approximate counting in the general Holant framework without a degree bound. We also formally introduce holographic reduction in the study of approximate counting and these weighted Fibonacci gate problems serve as computation primitives for approximate counting. Under holographic reduction, we obtain FPTAS for other Holant problems and spin problems. One important application is developing an FPTAS for a large range of ferromagnetic two-state spin systems. This is the first deterministic FPTAS in the ferromagnetic range for two-state spin systems without a degree bound. Besides these algorithms, we also develop several new tools and techniques to establish the correlation decay property, which are applicable in other problems.

cs.DS↗

arXiv2012

Approximate Counting via Correlation Decay on Planar Graphs

We show for a broad class of counting problems, correlation decay (strong spatial mixing) implies FPTAS on planar graphs. The framework for the counting problems considered by us is the Holant problems with arbitrary constant-size domain and symmetric constraint functions. We define a notion of regularity on the constraint functions, which covers a wide range of natural and important counting problems, including all multi-state spin systems, counting graph homomorphisms, counting weighted matchings or perfect matchings, the subgraphs world problem transformed from the ferromagnetic Ising model, and all counting CSPs and Holant problems with symmetric constraint functions of constant arity. The core of our algorithm is a fixed-parameter tractable algorithm which computes the exact values of the Holant problems with regular constraint functions on graphs of bounded treewidth. By utilizing the locally tree-like property of apex-minor-free families of graphs, the parameterized exact algorithm implies an FPTAS for the Holant problem on these graph families whenever the Gibbs measure defined by the problem exhibits strong spatial mixing. We further extend the recursive coupling technique to Holant problems and establish strong spatial mixing for the ferromagnetic Potts model and the subgraphs world problem. As consequences, we have new deterministic approximation algorithms on planar graphs and all apex-minor-free graphs for several counting problems.

cs.DS↗