arXiv subjects

Guoliang Qiu

Publications and source records attributed to Guoliang Qiu.

8 recordsLinked to original sources

arXiv2026

BAPS: A Fine-Grained Low-Precision Scheme for Softmax in Attention via Block-Aware Precision reScaling

As the performance gains from accelerating quantized matrix multiplication plateau, the softmax operation becomes the critical bottleneck in Transformer inference. This bottleneck stems from two hardware limitations: (1) limited data bandwidth between matrix and vector compute cores, and (2) the significant area cost of high-precision (FP32/16) exponentiation units (EXP2). To address these issues, we introduce a novel low-precision workflow that employs a specific 8-bit floating-point format (HiF8) and block-aware precision rescaling for softmax. Crucially, our algorithmic innovations make low-precision softmax feasible without the significant model accuracy loss that hampers direct low-precision approaches. Specifically, our design (i) halves the required data movement bandwidth by enabling matrix multiplication outputs constrained to 8-bit, and (ii) substantially reduces the EXP2 unit area by computing exponentiations in low (8-bit) precision. Extensive evaluation on language models and multi-modal models confirms the validity of our method. By alleviating the vector computation bottleneck, our work paves the way for doubling end-to-end inference throughput without increasing chip area, and offers a concrete co-design path for future low-precision hardware and software.

cs.LG↗

arXiv2025

Sampling permutations satisfying constraints within the lopsided local lemma regime

Sampling a random permutation with restricted positions, or equivalently approximating the permanent of a 0-1 matrix, is a fundamental problem in computer science, with several notable results achieved over the years. However, existing algorithms typically exhibit high computational complexity. Achieving the optimal running time remains elusive, even for nontrivial subsets of the problem. Furthermore, existing algorithms primarily focus on a single permutation, leaving many combinatorial problems involving multiple constrained permutations unaddressed. For a single permutation, we achieve the optimal running time $O(n^2)$ for approximating the permanent of a very dense $n \times n$ 0-1 matrix, where each row and column contains at most $\sqrt{(n-2)/20}$ zeros. This result serves as a fundamental building block in our sampling algorithm for multiple permutations. We further introduce a general model called permutations with disjunctive constraints (PDC) for handling multiple constrained permutations. We propose a novel Markov chain-based algorithm for sampling nearly uniform solutions of PDC within a lopsided Lovász Local Lemma (LLL) regime. For uniform PDC formulas, where all constraints are of the same width and all permutations are of the same size, our algorithm runs in nearly linear time with respect to the number of variables. Previous approaches for sampling LLL relied on the variable model. In contrast, the sampling problem of PDC encounters a fundamental challenge: the random variables within each permutation in the joint probability space are not mutually independent, leading to long-range correlations. To tackle this challenge, we introduce a novel sampling framework called correlated factorization and a new concept in the path coupling analysis, termed the inactive vertex.

cs.DS↗

arXiv2025

A Thorough Comparison Between Independent Cascade and Susceptible-Infected-Recovered Models

We study cascades in social networks with the independent cascade (IC) model and the Susceptible-Infected-recovered (SIR) model. The well-studied IC model fails to capture the feature of node recovery, and the SIR model is a variant of the IC model with the node recovery feature. In the SIR model, by computing the probability that a node successfully infects another before its recovery and viewing this probability as the corresponding IC parameter, the SIR model becomes an "out-going-edge-correlated" version of the IC model: the events of the infections along different out-going edges of a node become dependent in the SIR model, whereas these events are independent in the IC model. In this paper, we thoroughly compare the two models and examine the effect of this extra dependency in the SIR model. By a carefully designed coupling argument, we show that the seeds in the IC model have a stronger influence spread than their counterparts in the SIR model, and sometimes it can be significantly stronger. Specifically, we prove that, given the same network, the same seed sets, and the parameters of the two models being set based on the above-mentioned equivalence, the expected number of infected nodes at the end of the cascade for the IC model is weakly larger than that for the SIR model, and there are instances where this dominance is significant. We also study the influence maximization problem with the SIR model. We show that the above-mentioned difference in the two models yields different seed-selection strategies, which motivates the design of influence maximization algorithms specifically for the SIR model. We design efficient approximation algorithms with theoretical guarantees by adapting the reverse-reachable-set-based algorithms, commonly used for the IC model, to the SIR model. Finally, we conduct experimental studies over real-world datasets.

cs.SI↗

arXiv2024

FPTAS for Holant Problems with Log-Concave Signatures

For an integer $b\ge 0$, a $b$-matching in a graph $G=(V,E)$ is a set $S\subseteq E$ such that each vertex $v\in V$ is incident to at most $b$ edges in $S$. We design a fully polynomial-time approximation scheme (FPTAS) for counting the number of $b$-matchings in graphs with bounded degrees. Our FPTAS also applies to a broader family of counting problems, namely Holant problems with log-concave signatures. Our algorithm is based on Moitra's linear programming approach (JACM'19). Using a novel construction called the extended coupling tree, we derandomize the coupling designed by Chen and Gu (SODA'24).

cs.DS↗

arXiv2023

Improved Competitive Ratio for Edge-Weighted Online Stochastic Matching

We consider the edge-weighted online stochastic matching problem, in which an edge-weighted bipartite graph G=(I\cup J, E) with offline vertices J and online vertex types I is given. The online vertices have types sampled from I with probability proportional to the arrival rates of online vertex types. The online algorithm must make immediate and irrevocable matching decisions with the objective of maximizing the total weight of the matching. For the problem with general arrival rates, Feldman et al. (FOCS 2009) proposed the Suggested Matching algorithm and showed that it achieves a competitive ratio of 1-1/e \approx 0.632. The ratio has recently been improved to 0.645 by Yan (2022), who proposed the Multistage Suggested Matching (MSM) algorithm. In this paper, we propose the Evolving Suggested Matching (ESM) algorithm, and show that it achieves a competitive ratio of 0.650.

cs.DS↗

arXiv2023

Inapproximability of Counting Independent Sets in Linear Hypergraphs

It is shown in this note that approximating the number of independent sets in a $k$-uniform linear hypergraph with maximum degree at most $Δ$ is NP-hard if $Δ\geq 5\cdot 2^{k-1}+1$. This confirms that for the relevant sampling and approximate counting problems, the regimes on the maximum degree where the state-of-the-art algorithms work are tight, up to some small factors. These algorithms include: the approximate sampler and randomised approximation scheme by Hermon, Sly and Zhang (RSA, 2019), the perfect sampler by Qiu, Wang and Zhang (ICALP, 2022), and the deterministic approximation scheme by Feng, Guo, Wang, Wang and Yin (FOCS, 2023).

cs.CC↗

arXiv2023

MMS Allocations of Chores with Connectivity Constraints: New Methods and New Results

We study the problem of allocating indivisible chores to agents under the Maximin share (MMS) fairness notion. The chores are embedded on a graph and each bundle of chores assigned to an agent should be connected. Although there is a simple algorithm for MMS allocations of goods on trees, it remains open whether MMS allocations of chores on trees always exist or not, which is a simple but annoying problem in chores allocation. In this paper, we introduce a new method for chores allocation with connectivity constraints, called the group-satisfied method, that can show the existence of MMS allocations of chores on several subclasses of trees. Even these subcases are non-trivial and our results can be considered as a significant step to the open problem. We also consider MMS allocations of chores on cycles where we get the tight approximation ratio for three agents. Our result was obtained via the linear programming (LP) method, which enables us not only to compute approximate MMS allocations but also to construct tight examples of the nonexistence of MMS allocations without complicated combinatorial analysis. These two proposed methods, the group-satisfied method and the LP method, have the potential to solve more related problems.

cs.DS↗

arXiv2022

A Perfect Sampler for Hypergraph Independent Sets

The problem of uniformly sampling hypergraph independent sets is revisited. We design an efficient perfect sampler for the problem under a condition similar to that of the asymmetric Lovász Local Lemma. When applied to $d$-regular $k$-uniform hypergraphs on $n$ vertices, our sampler terminates in expected $O(n\log n)$ time provided $d\le c\cdot 2^{k/2}/k$ for some constant $c>0$. If in addition the hypergraph is linear, the condition can be weaken to $d\le c\cdot 2^{k}/k^2$ for some constant $c>0$, matching the rapid mixing condition for Glauber dynamics in Hermon, Sly and Zhang [HSZ19].

cs.DS↗