Searcharxiv⌕ Search

arXiv subjects

R. W. R. Darling

Publications and source records attributed to R. W. R. Darling.

At least 19 recordsLinked to original sources
arXiv2026

Rank-based linkage I: triplet comparisons and oriented simplicial complexes

Rank-based linkage is a new tool for summarizing a collection $S$ of objects according to their relationships. These objects are not mapped to vectors, and ``similarity'' between objects need be neither numerical nor symmetrical. All an object needs to do is rank nearby objects by similarity to itself, using a Comparator which is transitive, but need not be consistent with any metric on the whole set. Call this a ranking system on $S$. Rank-based linkage is applied to the $K$-nearest neighbor digraph derived from a ranking system. Computations occur on a 2-dimensional abstract oriented simplicial complex whose faces are among the points, edges, and triangles of the line graph of the undirected $K$-nearest neighbor graph on $S$. In $|S| K^2$ steps it builds an edge-weighted linkage graph $(S, \mathcal{L}, σ)$ where $σ(\{x, y\})$ is called the in-sway between objects $x$ and $y$. Take $\mathcal{L}_t$ to be the links whose in-sway is at least $t$, and partition $S$ into components of the graph $(S, \mathcal{L}_t)$, for varying $t$. Rank-based linkage is a functor from a category of ``out-ordered'' digraphs to a category of partitioned sets, with the practical consequence that augmenting the set of objects in a rank-respectful way gives a fresh clustering which does not ``rip apart'' the previous one. The same holds for single linkage clustering in the metric space context, but not for typical optimization-based methods. Orientation sheaves play in a fundamental role and ensure that partially overlapping data sets can be ``glued'' together. Open combinatorial problems are presented in the last section.

math.CO↗
arXiv2025

Prüfer codes on vertex-colored rooted trees

Prüfer codes provide an encoding scheme for representing a vertex-labeled tree on $n$ vertices with a string of length $n-2$. Indeed, two labeled trees are isomorphic if and only if their Prüfer codes are identical, and this supplies a proof of Cayley's Theorem. Motivated by a graph decomposition of freight networks into a corpus of vertex-colored rooted trees, we extend the notion of Prüfer codes to that setting, i.e., trees without a unique labeling, by defining a canonical label for a vertex-colored rooted tree and incorporating vertex colors into our variation of the Prüfer code. Given a pair of trees, we prove properties of the vertex-colored Prüfer code (abbreviated VCPC) equivalent to (1) isomorphism between a pair of vertex-colored rooted trees, (2) the subtree relationship between vertex-colored rooted trees, and (3) when one vertex-colored rooted tree is isomorphic to a minor of another vertex-colored rooted tree.

math.CO↗
arXiv2023

Hidden Ancestor Graphs: Models for Detagging Property Graphs

Consider a graph $G$ where each vertex is visibly labelled as a member of a distinct class, but also has a hidden binary state: wild or tame. Edges with end points in the same class are called agreement edges. Premise: an edge connecting vertices in different classes -- a conflict edge -- is allowed only when at least one end point is wild. Interpret wild status as readiness to form connections with any other vertex, regardless of class -- a form of class disaffiliation. The learning goal is to classify each vertex as wild or tame using its neighborhood data. In applications such as communications metadata, bio-informatics, retailing, or bibliography, adjacency in $G$ is typically created by paths of length two in a transactional bipartite graph $B$. Class labelling, imported from a reference data source, is typically assortative, so agreement edges predominate. Conflict edges represent observed behavior (from $B$) inconsistent with prior labelling of $V(G)$. Wild vertices are those whose label is uninformative. The hidden ancestor graph constitutes a natural model for generating agreement edges and conflict edges, depending on a latent tree structure. The model is able to manifest high clustering rates and heavy-tailed degree distributions typical of social and spatial networks. It can be fitted to graph data using a few measurable graph parameters, and supplies a natural statistical classifier for wild versus tame.

math.PR↗
arXiv2022

Proceedings of TDA: Applications of Topological Data Analysis to Data Science, Artificial Intelligence, and Machine Learning Workshop at SDM 2022

Topological Data Analysis (TDA) is a rigorous framework that borrows techniques from geometric and algebraic topology, category theory, and combinatorics in order to study the "shape" of such complex high-dimensional data. Research in this area has grown significantly over the last several years bringing a deeply rooted theory to bear on practical applications in areas such as genomics, natural language processing, medicine, cybersecurity, energy, and climate change. Within some of these areas, TDA has also been used to augment AI and ML techniques. We believe there is further utility to be gained in this space that can be facilitated by a workshop bringing together experts (both theorists and practitioners) and non-experts. Currently there is an active community of pure mathematicians with research interests in developing and exploring the theoretical and computational aspects of TDA. Applied mathematicians and other practitioners are also present in community but do not represent a majority. This speaks to the primary aim of this workshop which is to grow a wider community of interest in TDA. By fostering meaningful exchanges between these groups, from across the government, academia, and industry, we hope to create new synergies that can only come through building a mutual comprehensive awareness of the problem and solution spaces.

math.AT↗
arXiv2022

Empirical complexity of comparator-based nearest neighbor descent

A Java parallel streams implementation of the $K$-nearest neighbor descent algorithm is presented using a natural statistical termination criterion. Input data consist of a set $S$ of $n$ objects of type V, and a Function >, which enables any $x \in S$ to decide which of $y, z \in S\setminus\{x\}$ is more similar to $x$. Experiments with the Kullback-Leibler divergence Comparator support the prediction that the number of rounds of $K$-nearest neighbor updates need not exceed twice the diameter of the undirected version of a random regular out-degree $K$ digraph on $n$ vertices. Overall complexity was $O(n K^2 \log_K(n))$ in the class of examples studied. When objects are sampled uniformly from a $d$-dimensional simplex, accuracy of the $K$-nearest neighbor approximation is high up to $d = 20$, but declines in higher dimensions, as theory would predict.

cs.LG↗
arXiv2021

Partitioned K-nearest neighbor local depth for scalable comparison-based learning

A triplet comparison oracle on a set $S$ takes an object $x \in S$ and for any pair $\{y, z\} \subset S \setminus \{x\}$ declares which of $y$ and $z$ is more similar to $x$. Partitioned Local Depth (PaLD) supplies a principled non-parametric partitioning of $S$ under such triplet comparisons but needs $O(n^2 \log{n})$ oracle calls and $O(n^3)$ post-processing steps. We introduce Partitioned Nearest Neighbors Local Depth (PaNNLD), a computationally tractable variant of PaLD leveraging the $K$-nearest neighbors digraph on $S$. PaNNLD needs only $O(n K \log{n})$ oracle calls, by replacing an oracle call by a coin flip when neither $y$ nor $z$ is adjacent to $x$ in the undirected version of the $K$-nearest neighbors digraph. By averaging over randomizations, PaNNLD subsequently requires (at best) only $O(n K^2)$ post-processing steps. Concentration of measure shows that the probability of randomization-induced error $δ$ in PaNNLD is no more than $2 e^{-δ^2 K^2}$.

cs.DS↗
arXiv2020

K-Nearest Neighbor Approximation Via the Friend-of-a-Friend Principle

Suppose $V$ is an $n$-element set where for each $x \in V$, the elements of $V \setminus \{x\}$ are ranked by their similarity to $x$. The $K$-nearest neighbor graph is a directed graph including an arc from each $x$ to the $K$ points of $V \setminus \{x\}$ most similar to $x$. Constructive approximation to this graph using far fewer than $n^2$ comparisons is important for the analysis of large high-dimensional data sets. $K$-Nearest Neighbor Descent is a parameter-free heuristic where a sequence of graph approximations is constructed, in which second neighbors in one approximation are proposed as neighbors in the next. Run times in a test case fit an $O(n K^2 \log{n})$ pattern. This bound is rigorously justified for a similar algorithm, using range queries, when applied to a homogeneous Poisson process in suitable dimension. However the basic algorithm fails to achieve subquadratic complexity on sets whose similarity rankings arise from a ``generic'' linear order on the $\binom{n}{2}$ inter-point distances in a metric space.

math.CO↗
arXiv2018

Anomaly Detection and Correction in Large Labeled Bipartite Graphs

Binary classification problems can be naturally modeled as bipartite graphs, where we attempt to classify right nodes based on their left adjacencies. We consider the case of labeled bipartite graphs in which some labels and edges are not trustworthy. Our goal is to reduce noise by identifying and fixing these labels and edges. We first propose a geometric technique for generating random graph instances with untrustworthy labels and analyze the resulting graph properties. We focus on generating graphs which reflect real-world data, where degree and label frequencies follow power law distributions. We review several algorithms for the problem of detection and correction, proposing novel extensions and making observations specific to the bipartite case. These algorithms range from math programming algorithms to discrete combinatorial algorithms to Bayesian approximation algorithms to machine learning algorithms. We compare the performance of all these algorithms using several metrics and, based on our observations, identify the relative strengths and weaknesses of each individual algorithm.

math.CO↗
arXiv2018

The Combinatorial Data Fusion Problem in Conflicted-supervised Learning

The best merge problem in industrial data science generates instances where disparate data sources place incompatible relational structures on the same set $V$ of objects. Graph vertex labelling data may include (1) missing or erroneous labels,(2) assertions that two vertices carry the same (unspecified) label, and (3) denying some subset of vertices from carrying the same label. Conflicted-supervised learning applies to cases where no labelling scheme satisfies (1), (2), and (3). Our rigorous formulation starts from a connected weighted graph $(V, E)$, and an independence system $\mathcal{S}$ on $V$, characterized by its circuits, called forbidden sets. Global incompatibility is expressed by the fact $V \notin \mathcal{S}$. Combinatorial data fusion seeks a subset $E_1 \subset E$ of maximum edge weight so that no vertex component of the subgraph $(V, E_1)$ contains any forbidden set. Multicut and multiway cut are special cases where all forbidden sets have cardinality two. The general case exhibits unintuitive properties, shown in counterexamples. The first in a series of papers concentrates on cases where $(V, E)$ is a tree, and presents an algorithm on general graphs, in which the combinatorial data fusion problem is transferred to the Gomory-Hu tree, where it is solved using greedy set cover. Experimental results are given.

math.CO↗
arXiv2012

Rank deficiency in sparse random GF[2] matrices

Let $M$ be a random $m \times n$ matrix with binary entries and i.i.d. rows. The weight (i.e., number of ones) of a row has a specified probability distribution, with the row chosen uniformly at random given its weight. Let $N(n,m)$ denote the number of left null vectors in ${0,1}^m$ for $M$ (including the zero vector), where addition is mod 2. We take $n, m \to \infty$, with $m/n \to α> 0$, while the weight distribution may vary with $n$ but converges weakly to a limiting distribution on ${3, 4, 5, ...}$; let $W$ denote a variable with this limiting distribution. Identifying $M$ with a hypergraph on $n$ vertices, we define the 2-core of $M$ as the terminal state of an iterative algorithm that deletes every row incident to a column of degree 1. We identify two thresholds $α^*$ and $\underlineα$, and describe them analytically in terms of the distribution of $W$. Threshold $α^*$ marks the infimum of values of $α$ at which $n^{-1} \log{\mathbb{E} [N(n,m)}]$ converges to a positive limit, while $\underlineα$ marks the infimum of values of $α$ at which there is a 2-core of non-negligible size compared to $n$ having more rows than non-empty columns. We have $1/2 \leq α^* \leq \underlineα \leq 1$, and typically these inequalities are strict; for example when $W = 3$ almost surely, numerics give $α^* = 0.88949 ...$ and $\underlineα = 0.91793 ...$ (previous work on this model has mainly been concerned with such cases where $W$ is non-random). The threshold of values of $α$ for which $N(n,m) \geq 2$ in probability lies in $[α^*,\underlineα]$ and is conjectured to equal $\underlineα$. The random row weight setting gives rise to interesting new phenomena not present in the non-random case that has been the focus of previous work.

math.PR↗
arXiv2009

Maximum GCD Among Pairs of Random Integers

Fix $α>0$, and sample $N$ integers uniformly at random from $\{1,2,\ldots ,\lfloor e^{αN}\rfloor \}$. Given $η>0$, the probability that the maximum of the pairwise GCDs lies between $N^{2-η}$ and $N^{2+η}$ converges to 1 as $N\to \infty $. More precise estimates are obtained. This is a Birthday Problem: two of the random integers are likely to share some prime factor of order $N^2/\log [N]$. The proof generalizes to any arithmetical semigroup where a suitable form of the Prime Number Theorem is valid.

math.NT↗
arXiv2008

Differential equation approximations for Markov chains

We formulate some simple conditions under which a Markov chain may be approximated by the solution to a differential equation, with quantifiable error probabilities. The role of a choice of coordinate functions for the Markov chain is emphasised. The general theory is illustrated in three examples: the classical stochastic epidemic, a population process model with fast and slow variables, and core-finding algorithms for large random hypergraphs.

math.PR↗
arXiv2005

Structure of large random hypergraphs

The theme of this paper is the derivation of analytic formulae for certain large combinatorial structures. The formulae are obtained via fluid limits of pure jump-type Markov processes, established under simple conditions on the Laplace transforms of their Levy kernels. Furthermore, a related Gaussian approximation allows us to describe the randomness which may persist in the limit when certain parameters take critical values. Our method is quite general, but is applied here to vertex identifiability in random hypergraphs. A vertex v is identifiable in n steps if there is a hyperedge containing v all of whose other vertices are identifiable in fewer steps. We say that a hyperedge is identifiable if every one of its vertices is identifiable. Our analytic formulae describe the asymptotics of the number of identifiable vertices and the number of identifiable hyperedges for a Poisson(β) random hypergraph Λon a set V of N vertices, in the limit as N\to \infty. Here βis a formal power series with nonnegative coefficients β_0,β_1,..., and (Λ(A))_{A\subseteq V} are independent Poisson random variables such that Λ(A), the number of hyperedges on A, has mean Nβ_j/\pmatrixN j whenever |A|=j.

math.PR↗
arXiv2004

Continuous and discontinuous phase transitions in hypergraph processes

Let V denote a set of N vertices. To construct a "hypergraph process", create a new hyperedge at each event time of a Poisson process; the cardinality K of this hyperedge is random, with arbitrary probability generating function r(x), except that we assume P(K=1) +P(K=2) > 0. Given K=k, the k vertices appearing in the new hyperedge are selected uniformly at random from V. Hyperedges of cardinality 1 are called patches, and serve as a way of selecting root vertices. Identifiable vertices are those which are reachable from these root vertices, in a strong sense which generalizes the notion of graph component. Hyperedges are also called identifiable if all of their vertices are identifiable. We use "fluid limit" scaling: hyperedges arrive at rate N, and we study structures of size O(1) and O(N). After division by N, numbers of identifiable vertices and reducible hyperedges exhibit phase transitions, which may be continuous or discontinuous depending on the shape of the structure function -log(1 - x)/r'(x), for x in (0,1). Both the case P(K=1) > 0 and the case P(K=1) = 0 < P(K=2) are considered; for the latter, a single extraneous patch is added to mark the root vertex.

math.PR↗
arXiv2004

Structure of large random hypergraphs

The theme of this paper is the derivation of analytic formulae for certain large combinatorial structures. The formulae are obtained via fluid limits of pure jump type Markov processes, established under simple conditions on the Laplace transforms of their Levy kernels. Furthermore, a related Gaussian approximation allows us to describe the randomness which may persist in the limit when certain parameters take critical values. Our method is quite general, but is applied here to vertex identifiability in random hypergraphs. A vertex v is identifiable in n steps if there is a hyperedge containing v all of whose other vertices are identifiable in fewer than n steps. We say that a hyperedge is identifiable if every one of its vertices is identifiable. Our analytic formulae describe the asymptotics of the number of identifiable vertices and the number of identifiable hyperedges for a Poisson random hypergraph on a set of N vertices, in the limit as N goes to infinity.

math.PR↗
arXiv2002

Fluid Limits of Pure Jump Markov Processes: a Practical Guide

A rescaled Markov chain converges uniformly in probability to the solution of an ordinary differential equation, under carefully specified assumptions. The presentation is much simpler than those in the outside literature. The result may be used to build parsimonious models of large random or pseudo-random systems.

math.PR↗
arXiv1998

The Repeated Solicitation Model

This paper presents a probabilistic analysis of what we call the "repeated solicitation model". To give a specific context, suppose B is a direct marketing company with a list of S sales prospects. At epoch 1, B sends a solicitation to every prospect on the list, and elicits X(1) replies. The company deletes the respondents from the list, and at epoch 2 sends a solicitation to the other prospects, of whom X(2) respond, and so on. This continues until an epoch n such that X(n) = 0, which we call epoch T, and then B makes no further solicitations. We seek (a) the probability distribution of T; (b) the distribution of the total number of respondents; (c) the expected total number of solicitations. All three quantities are explicitly computed, assuming that (i) prospects' response times are independent, and (ii) S is Poisson distributed.

math.PR↗