arXiv subjects

Thomas Griffin

Publications and source records attributed to Thomas Griffin.

6 recordsLinked to original sources

arXiv2026

Filtered Vector Search in a Disaggregated Lakehouse: Composing Table-Format Pruning with Per-File ANN

Approximate nearest-neighbor (ANN) search increasingly runs alongside structured data - "find the 10 nearest documents where tenant='acme' AND lang='en'" - yet similarity and filtering are usually bolted together: a specialized vector index for one, a separate filter step for the other. We ask what happens when both live inside an open lakehouse table (Apache Iceberg over Parquet on object storage), where the engine already owns a mature file-pruning stack (partition pruning, zone-maps, a bitmap index). We embed an IVF index in place in each Parquet file's footer and make filtered vector queries fast not with a new filtering algorithm but by composing the table's existing file pruning with per-file ANN: the planner prunes data files by the predicate first, then runs IVF only over the survivors. The index is built distributed and non-destructively - a metadata-only Iceberg replace that every other engine still reads - and a rendezvous-hashed per-file cache keeps object-store read latency from swamping the algorithmic win. The payoff comes entirely from file pruning. On an 11.5M x 768 table, warm IVF search is ~32x faster than brute force at recall@10 >= 0.90, a selective predicate having pruned 355 of 444 data files before ANN runs; on 5M real IBM Granite embeddings, a filter arriving across a join prunes four of five region partitions and runs nearly two orders of magnitude (~94x: 14.7 s -> 157 ms) faster than the query-time join at identical top-k, once the reduction is materialized into a region-partitioned layout. We characterize when the composition pays off - it requires file-level locality on the filter column, and the residual predicate is only safe to push into the search over a provably pure (partitioned) column, not a merely sorted one - and report the failure modes we hit bolting ANN onto a lakehouse engine.

cs.DB↗

arXiv2024

A modified chemostat exhibiting competitive exclusion "reversal"

The classical chemostat is an intensely investigated model in ecology and bio/chemical engineering, where n-species, say $x_{1}, x_{2}...x_{n}$ compete for a single growth limiting nutrient. Classical theory predicts that depending on model parameters, one species competitively excludes all others. Furthermore, this ''order'' of strongest to weakest is preserved, $x_{1} >> x_{2} >> ...x_{n}$, for say $D_{1} < D_{2} <...D_{n}$, where $D_{i}$ is the net removal of species $x_{i}$. Meaning $x_{1}$ is the strongest or most dominant species and $x_{n}$ is the weakest or least dominant. We propose a modified version of the classical chemostat, exhibiting certain counterintuitive dynamics. Herein we show that if only a certain proportion of the weakest species $x_{n}$'s population is removed at a ''very'' fast density dependent rate, it will in fact be able to competitively exclude all other species, for certain initial conditions. Numerical simulations are carried out to visualize these dynamics in the three species case.

math.DS↗

arXiv2023

Cutoff in the Bernoulli-Laplace Model With Unequal Colors and Urn Sizes

We consider a generalization of the Bernoulli-Laplace model in which there are two urns and $n$ total balls, of which $r$ are red and $n - r$ white, and where the left urn holds $m$ balls. At each time increment, $k$ balls are chosen uniformly at random from each urn and then swapped. This system can be used to model phenomena such as gas particle interchange between containers or card shuffling. Under a reasonable set of assumptions, we bound the mixing time of the resulting Markov chain asymptotically in $n$ with cutoff at $\log{n}$ and constant window. Among other techniques, we employ the spectral analysis of arXiv:0906.4242 on the Markov transition kernel and the chain coupling tools of arXiv:2203.08647 and arXiv:1606.01437.

math.PR↗

arXiv2019

Large and Small Data Blow-Up Solutions in the Trojan Y Chromosome Model

The Trojan Y Chromosome Strategy (TYC) is an extremely well investigated biological control method for controlling invasive populations with an XX-XY sex determinism. In \cite{GP12, WP14} various dynamical properties of the system are analyzed, including well posedness, boundedness of solutions, and conditions for extinction or recovery. These results are derived under the assumption of positive solutions. In the current manuscript, we show that if the introduction rate of trojan fish is zero, under certain large data assumptions, negative solutions are possible for the male population, which in turn can lead to finite time blow-up in the female and male populations. A comparable result is established for \emph{any} positive initial condition if the introduction rate of trojan fish is large enough. Similar finite time blow-up results are obtained in a spatial temporal TYC model that includes diffusion. Lastly, we investigate improvements to the TYC modeling construct that may dampen the mechanisms to the blow-up phenomenon or remove the negativity of solutions. The results draw into suspect the reliability of current TYC models under certain situations.

math.DS↗

arXiv2019

Optimal Control and Analysis of a Modified Trojan Y-Chromosome Strategy

The Trojan Y Chromosome (TYC) Strategy is a promising eradication method that attempts to manipulate the female to male ratio to promote the reduction of the population of an invasive species. The manipulation stems from an introduction of sex-reversed males, called supermales, into an ecosystem. The offspring of the supermales is guaranteed to be male. Mathematical models have shown that the population can be driven to extinction with a continuous supply of supermales. In this paper, a new model of the TYC strategy is introduced and analyzed that includes two important modeling characteristics, that are neglected in all previous models. First, the new model includes intraspecies competition for mates. Second, a strong Allee effect is included. Several conclusions about the strategy via optimal control are established. These results have large scale implications for the biological control of invasive species.

q-bio.PE↗

arXiv2018

Data Infrastructure and Approaches for Ontology-Based Drug Repurposing

We report development of a data infrastructure for drug repurposing that takes advantage of two currently available chemical ontologies. The data infrastructure includes a database of compound- target associations augmented with molecular ontological labels. It also contains two computational tools for prediction of new associations. We describe two drug-repurposing systems: one, Nascent Ontological Information Retrieval for Drug Repurposing (NOIR-DR), based on an information retrieval strategy, and another, based on non-negative matrix factorization together with compound similarity, that was inspired by recommender systems. We report the performance of both tools on a drug-repurposing task.

cs.IR↗