arXiv subjects

Tobias Rubel

Publications and source records attributed to Tobias Rubel.

3 recordsLinked to original sources

arXiv2026

PiPNN: Ultra-Scalable Graph-Based Nearest Neighbor Indexing

The fastest indexes for Approximate Nearest Neighbor Search today are also the slowest to build: graph-based methods like HNSW and Vamana achieve state-of-the-art query performance but have large construction times due to relying on random-access-heavy beam searches. We introduce PiPNN (Pick-in-Partitions Nearest Neighbors), an ultra-scalable graph construction algorithm that avoids this ``search bottleneck'' that existing graph-based methods suffer from. PiPNN's core innovation is HashPrune, a novel online pruning algorithm which dynamically maintains sparse collections of edges. HashPrune enables PiPNN to partition the dataset into overlapping sub-problems, efficiently perform bulk distance comparisons via dense matrix multiplication kernels, and stream a subset of the edges into HashPrune. HashPrune guarantees bounded memory during index construction which permits PiPNN to build higher quality indices without the use of extra intermediate memory. PiPNN builds state-of-the-art indexes up to 11.6x faster than Vamana (DiskANN) and up to 12.9x faster than HNSW. PiPNN is significantly more scalable than recent algorithms for fast graph construction. PiPNN builds indexes at least 19.1x faster than MIRAGE and 17.3x than FastKCNA while producing indexes that achieve higher query throughput. PiPNN enables us to build, for the first time, high-quality ANN indexes on billion-scale datasets in under 20 minutes using a single multicore machine.

cs.DB↗

arXiv2024

Graphery: Interactive Tutorials for Biological Network Algorithms

Networks provide a meaningful way to represent and analyze complex biological information, but the methodological details of network-based tools are often described for a technical audience. Graphery is a hands-on tutorial webserver designed to help biological researchers understand the fundamental concepts behind commonly-used graph algorithms. Each tutorial describes a graph concept along with executable Python code that visualizes the concept in a code view and a graph view. Graphery tutorials help researchers understand graph statistics (such as degree distribution and network modularity) and classic graph algorithms (such as shortest paths and random walks). Users navigate each tutorial using their choice of real-world biological networks, ranging in scale from molecular interaction graphs to ecological networks. Graphery also allows users to modify the code within each tutorial or write new programs, which all can be executed without requiring an account. Discipline-focused tutorials will be essential to help researchers interpret their biological data. Graphery accepts ideas for new tutorials and datasets that will be shaped by both computational and biological researchers, growing into a community-contributed learning platform. Availability: Graphery is available at https://graphery.reedcompbio.org/.

q-bio.MN↗

arXiv2021

Dropping diversity of products of large US firms: Models and measures

It is widely assumed that in our lifetimes the products available in the global economy have become more diverse. This assumption is difficult to investigate directly, however, because it is difficult to collect the necessary data about every product in an economy each year. We solve this problem by mining publicly available textual descriptions of the products of every large US firms each year from 1997 to 2017. Although many aspects of economic productivity have been steadily rising during this period, our text-based measurements show that the diversity of the products of at least large US firms has steadily declined. This downward trend is visible using a variety of product diversity metrics, including some that depend on a measurement of the similarity of the products of every single pair of firms. The current state of the art in comprehensive and detailed firm-similarity measurements is a Boolean word vector model due to Hoberg and Phillips. We measure diversity using firm-similarities from this Boolean model and two more sophisticated variants, and we consistently observe a significant dropping trend in product diversity. These results make it possible to frame and start to test specific hypotheses for explaining the dropping product diversity trend.

q-fin.ST↗