Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 0709.0670

Using Data Compressors to Construct Rank Tests

Daniil Ryabko·Juergen Schmidhuber
View original source ↗Read source document ↗Export metadata ↓

Abstract

Nonparametric rank tests for homogeneity and component independence are proposed, which are based on data compressors. For homogeneity testing the idea is to compress the binary string obtained by ordering the two joint samples and writing 0 if the element is from the first sample and 1 if it is from the second sample and breaking ties by randomization (extension to the case of multiple samples is straightforward). $H_0$ should be rejected if the string is compressed (to a certain degree) and accepted otherwise. We show that such a test obtained from an ideal data compressor is valid against all alternatives. Component independence is reduced to homogeneity testing by constructing two samples, one of which is the first half of the original and the other is the second half with one of the components randomly permuted.

Explore related subjects

cs.DScs.ITmath.IT

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Daniil Ryabko, Juergen Schmidhuber. 2007-09-05. Using Data Compressors to Construct Rank Tests. https://arxiv.org/abs/0709.0670

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Reweighted Spectral Partitioning Works: A Simple Algorithm for Vertex Separators in Special Graph Classes

We establish that a practical polynomial-time algorithm obtains near-optimal $\frac23$-balanced vertex separators for a number of graph classes, including genus-$g$ graphs, $K_h$-minor-free graphs, and geometric graphs. For many of these classes, our polynomial-time separator bounds improve on previous work. Moreover, unlike previous work, all these separators are produced by a single unified algorithm that has no specialization for the graph class, and this algorithm admits a scalable implementation with good empirical results. Furthermore, our approach is robust to graphs falling just outside these classes, making it appropriate for practical applications like road networks. A key distinction of our approach, compared to previous work, is the disentanglement of our algorithm and its analysis for specific graph classes. Our algorithm is based on a newly improved form of a recent Cheeger-style inequality relating the value of a special semidefinite program to vertex expansion. Our analysis then only has to directly bound the value of this semidefinite program for each graph class, and constructions for these bounds need not be algorithmic. Finally, we obtain several results of independent interest. These include an improved (existential) separator theorem for intersection graphs of $k$-ply $d$-dimensional balls, an improved bound on the Fiedler value of genus-$g$ graphs, and a complete spectral proof of the planar separator theorem.

cs.DS↗
arXiv2026

Quality control in sublinear time: a case study via random graphs

Many algorithms are designed to work well on average over inputs. But when such an algorithm is run on an arbitrary input, the question arises: Can we trust the algorithm on this input? This question is especially relevant in network science, where algorithms for theoretical random graph models are often applied to individual real-world networks. We identify a new class of algorithmic problems addressing this, which we call "Quality Control Problems." These problems are specified by a "quality function" $ρ$ and a distribution $D$ such that, with high probability, a sample drawn from $D$ is "high quality," meaning its $ρ$-value is near $1$. The goal is to accept most inputs $x \sim D$ and reject potentially adversarially generated inputs $x$ with $ρ(x)$ far from $1$. In this work, we consider the sublinear version of the quality control problem, where $D \in Δ(\{0,1\}^N)$ and the goal is to solve the $(D,ρ)$-quality problem with $o(N)$ queries and time. As a case study, we consider random graphs, i.e., $D = G_{n,p}$ (and $N = \binom{n}2$), and the $k$-clique count function $ρ_k := C_k(G)/\mathbb{E}_{G' \sim G_{n,p}}[C_k(G')]$, where $C_k(G)$ is the number of $k$-cliques in $G$. In suitable parameter ranges, worst-case approximate $k$-clique counting requires $p^{-Ω(k^2)}$ queries. In contrast, we show that the quality control problem for $G_{n,p}$ (with $n\geq p^{-ck}$ for some constant $c$) with respect to $ρ_k$ can be tested with $p^{-O(k)}$ queries and time, showing quality control is provably superpolynomially more efficient in this setting. More generally, for a motif $H$ of maximum degree $Δ(H)$, we show that the corresponding quality control problem can be solved with $p^{-O(Δ(H))}$ queries and running time.

cs.DS↗
arXiv2026

Deterministic Volume Estimation of Truncated Hypercubes

We present a deterministic polynomial-time algorithm for estimating the volume of a hypercube intersected by a fixed number of constraints, where each constraint is the sum of univariate functions that are each nonnegative, monotone, and convex. Such constraints include knapsack and norm-ball constraints. The case of the unit hypercube truncated by a single linear constraint (halfspace) is already #P-hard. Given a fixed number of such constraints in a given dimension, together with an error parameter, our algorithm computes a multiplicative approximation of the volume of their intersection with the unit hypercube in time polynomial in the dimension, reciprocal error, input encoding length, and output encoding length.

cs.DS↗