arXiv subjects

John Terilla

Publications and source records attributed to John Terilla.

At least 19 recordsLinked to original sources

arXiv2026

A calculus of types in Isbell nuclei

We identify two constructions from different mathematical traditions. In linear logic and realisability, logical types are generated rather than fixed in advance: one begins with a universe of realisers equipped with execution, uses orthogonality to test their interactions, and takes types to be the biorthogonally closed subsets. In enriched Isbell duality, a quantitative relation induces an adjunction whose fixed points form a category, its nucleus. These constructions proceed by different means; we show that, in the present setting, they produce the same objects. The shared datum is minimal: an associative product, called execution, and a real-valued measurement, with no compatibility assumed between them. The failure of the measurement to be additive is at once the relation defining orthogonality and the quantitative relation whose Isbell nucleus we form, and the types cut out by orthogonality are exactly the fixed points of the associated adjunction. The identification pays off in both directions. The most natural product of types fails to be associative; repairing this failure forces a different notion of type, sensitive to both sides of a composite, on which the induced product is associative and, when execution has units, carries two residuals. What emerges is a noncommutative Lambek calculus, derived directly from execution and orthogonality rather than imposed. In the reverse direction, each such type, read on the categorical side, generates a quantitative relation of its own, and with it a derived adjunction and a further generation of types; these derived types are again types of the original situation, computed by the residuals of the Lambek calculus. We also prove a coherence theorem for the threefold arrangements of this construction and, in the finite-dimensional case, give explicit formulas for the product.

cs.LO↗

arXiv2026

Projective metric geometry of tropical nuclei: gap matrices, event loci, and order chambers

The tropical row span and column span of a real matrix are, from the polyhedral point of view, different objects living in different ambient spaces. These polytopes are known to be combinatorially isomorphic as polyhedral complexes; we prove that they are isometric under a Hilbert projective metric. We show that this isometry, along with a considerable amount of additional metric and polyhedral structure, is a direct consequence of a single categorical construction: the Isbell nucleus of the matrix, viewed as a profunctor enriched over the extended reals. The projective nucleus carries two canonical structures inherited from enrichment. The first is a Hilbert projective metric, with respect to which the Isbell conjugate maps are mutually inverse isometries -- this is the Isometry Theorem. The second is a polyhedral cell decomposition cut out by the Isbell inequalities, recovering the type decomposition of tropical convexity. These two structures are linked pointwise by the \emph{gap matrix}. The Events Theorem identifies each positive entry of the gap matrix with the exact projective distance to the locus where the corresponding inequality becomes tight: algebraic slack in the Isbell inequalities equals geometric distance to the cell walls. Thresholding the gap matrix at successive radii produces a constructible sheaf of formal concept lattice towers, extracting discrete algebraic structure from the continuous geometry at each point. In the square case there is generically a unique full-dimensional cell. The Centering Theorem identifies its Chebyshev center -- the point maximally insulated from all cell walls -- and shows that the optimal radius equals the minimum directed cycle mean of an associated digraph, connecting the projective geometry of the nucleus to the classical theory of optimal assignments.

math.AG↗

arXiv2025

The Foundations of Tokenization: Statistical and Computational Concerns

Tokenization - the practice of converting strings of characters from an alphabet into sequences of tokens over a vocabulary - is a critical step in the NLP pipeline. The use of token representations is widely credited with increased model performance but is also the source of many undesirable behaviors, such as spurious ambiguity or inconsistency. Despite its recognized importance as a standard representation method in NLP, the theoretical underpinnings of tokenization are not yet fully understood. In particular, the impact of tokenization on language model estimation has been investigated primarily through empirical means. The present paper contributes to addressing this theoretical gap by proposing a unified formal framework for representing and analyzing tokenizer models. Based on the category of stochastic maps, this framework enables us to establish general conditions for a principled use of tokenizers and, most importantly, the necessary and sufficient conditions for a tokenizer model to preserve the consistency of statistical estimators. In addition, we discuss statistical and computational concerns crucial for designing and implementing tokenizer models, such as inconsistency, ambiguity, finiteness, and sequentiality. The framework and results advanced in this paper contribute to building robust theoretical foundations for representations in neural language modeling that can inform future theoretical and empirical research.

cs.CL↗

arXiv2024

From Language Models over Tokens to Language Models over Characters

Modern language models are internally -- and mathematically -- distributions over $\it{token}$ strings rather than $\it{character}$ strings, posing numerous challenges for programmers building user applications on top of them. For example, if a prompt is specified as a character string, it must be tokenized before passing it to the token-level language model. Thus, the tokenizer and consequent processing are very sensitive to the specification of the prompt (e.g., whether the prompt ends with a space or not). This paper presents algorithms for converting token-level language models to character-level ones. We present both exact and approximate algorithms. In the empirical portion of the paper, we benchmark the practical runtime and approximation quality. Across four publicly available language models, we find that -- even with a small computation budget -- our method is able to accurately approximate the character-level distribution at reasonably fast speeds, and that a significant improvement in the language model's compression rate (bits/byte) is achieved.

cs.CL↗

arXiv2021

An enriched category theory of language: from syntax to semantics

State of the art language models return a natural language text continuation from any piece of input text. This ability to generate coherent text extensions implies significant sophistication, including a knowledge of grammar and semantics. In this paper, we propose a mathematical framework for passing from probability distributions on extensions of given texts, such as the ones learned by today's large language models, to an enriched category containing semantic information. Roughly speaking, we model probability distributions on texts as a category enriched over the unit interval. Objects of this category are expressions in language, and hom objects are conditional probabilities that one expression is an extension of another. This category is syntactical -- it describes what goes with what. Then, via the Yoneda embedding, we pass to the enriched category of unit interval-valued copresheaves on this syntactical category. This category of enriched copresheaves is semantic -- it is where we find meaning, logical operations such as entailment, and the building blocks for more elaborate semantic concepts.

math.CT↗

arXiv2021

Tensor Networks for Probabilistic Sequence Modeling

Tensor networks are a powerful modeling framework developed for computational many-body physics, which have only recently been applied within machine learning. In this work we utilize a uniform matrix product state (u-MPS) model for probabilistic modeling of sequence data. We first show that u-MPS enable sequence-level parallelism, with length-n sequences able to be evaluated in depth O(log n). We then introduce a novel generative algorithm giving trained u-MPS the ability to efficiently sample from a wide variety of conditional distributions, each one defined by a regular expression. Special cases of this algorithm correspond to autoregressive and fill-in-the-blank sampling, but more complex regular expressions permit the generation of richly structured data in a manner that has no direct analogue in neural generative models. Experiments on sequence modeling with synthetic and real text data show u-MPS outperforming a variety of baselines and effectively generalizing their predictions in the presence of limited data.

cs.LG↗

arXiv2019

Modeling Sequences with Quantum States: A Look Under the Hood

Classical probability distributions on sets of sequences can be modeled using quantum states. Here, we do so with a quantum state that is pure and entangled. Because it is entangled, the reduced densities that describe subsystems also carry information about the complementary subsystem. This is in contrast to the classical marginal distributions on a subsystem in which information about the complementary system has been integrated out and lost. A training algorithm based on the density matrix renormalization group (DMRG) procedure uses the extra information contained in the reduced densities and organizes it into a tensor network model. An understanding of the extra information contained in the reduced densities allow us to examine the mechanics of this DMRG algorithm and study the generalization error of the resulting model. As an illustration, we work with the even-parity dataset and produce an estimate for the generalization error as a function of the fraction of the dataset used in training.

quant-ph↗

arXiv2019

Probabilistic Modeling with Matrix Product States

Inspired by the possibility that generative models based on quantum circuits can provide a useful inductive bias for sequence modeling tasks, we propose an efficient training algorithm for a subset of classically simulable quantum circuit models. The gradient-free algorithm, presented as a sequence of exactly solvable effective models, is a modification of the density matrix renormalization group procedure adapted for learning a probability distribution. The conclusion that circuit-based models offer a useful inductive bias for classical datasets is supported by experimental results on the parity learning problem.

quant-ph↗

arXiv2017

Language as a matrix product state

We propose a statistical model for natural language that begins by considering language as a monoid, then representing it in complex matrices with a compatible translation invariant probability measure. We interpret the probability measure as arising via the Born rule from a translation invariant matrix product state.

cs.CL↗

arXiv2016

Homotopy probability theory on a Riemannian manifold and the Euler equation

Homotopy probability theory is a version of probability theory in which the vector space of random variables is replaced with a chain complex. A natural example extends ordinary probability theory on a finite volume Riemannian manifold M. In this example, initial conditions for fluid flow on M are identified with collections of homotopy random variables and solutions to the Euler equation are identified with homotopies between collections of homotopy random variables. Several ideas about using homotopy probability theory to study fluid flow are introduced.

math.AT↗

arXiv2014

Cones in homotopy probability theory

This note defines cones in homotopy probability theory and demonstrates that a cone over a space is a reasonable replacement for the space. The homotopy Gaussian distribution in one variable is revisited as a cone on the ordinary Gaussian.

math.PR↗

arXiv2014

Homotopy Probability Theory II

This is the second of two papers that introduce a deformation theoretic framework to explain and broaden a link between homotopy algebra and probability theory. This paper outlines how the framework can assist in the development of homotopy probability theory, where a vector space of random variables is replaced by a chain complex of random variables. This allows the principles of derived mathematics to participate in classical and noncommutative probability theory. A simple example is presented.

math.PR↗

arXiv2013

Homotopy Probability Theory I

This is the first of two papers that introduce a deformation theoretic framework to explain and broaden a link between homotopy algebra and probability theory. In this paper, cumulants are proved to coincide with morphisms of homotopy algebras. The sequel paper outlines how the framework presented here can assist in the development of homotopy probability theory, allowing the principles of derived mathematics to participate in classical and noncommutative probability theory.

math.PR↗

arXiv2011

Homotopy DG algebras induce homotopy BV algebras

Let TA denote the space underlying the tensor algebra of a vector space A. In this short note, we show that if A is a differential graded algebra, then TA is a differential Batalin-Vilkovisky algebra. Moreover, if A is an A-infinity algebra, then TA is a commutative BV-infinity algebra.

math.QA↗

arXiv2009

Algebras over Cobar(coFrob)

We show that a square zero, degree one element in W(V), the Weyl algebra on a vector space V, is equivalent to providing V with the structure of an algebra over the properad Cobar(coFrob), the properad arising from the cobar construction applied to the cofrobenius coproperad.

math.QA↗

arXiv2009

Quantum backgrounds and QFT

We introduce the concept of a quantum background and a functor QFT. In the case that the QFT moduli space is smooth formal, we construct a flat quantum superconnection on a bundle over QFT which defines algebraic structures relevant to correlation functions in quantum field theory. We go further and identify chain level generalizations of correlation functions which should be present in all quantum field theories.

math.QA↗

arXiv2008

Smoothness Theorem for Differential BV Algebras

Associated to a differential BV algebra are two differential graded Lie algebras: we call one classical and the other, which contains a formal h-bar parameter, quantum. The classical dgLa is always smooth formal. In this paper, we give necessary and sufficient conditions for the quantum dgLa to be smooth formal. These conditions are equivalent to the degeneration of a version of the noncommutative Hodge to de Rham spectral sequence. References added.

math.QA↗

arXiv2004

Thermodynamic interpretation of quantum error correcting criterion

Shanon's fundamental coding theorems relate classical information theory to thermodynamics. More recent theoretical work has been successful in relating quantum information theory to thermodynamics. For example, Schumacher proved a quantum version of Shannon's 1948 classical noiseless coding theorem. In this note, we extend the connection between quantum information theory and thermodynamics to include quantum error correction. There is a standard mechanism for describing errors that may occur during the transmission, storage, and manipulation of quantum information. One can formulate a criterion of necessary and sufficient conditions for the errors to be detectable and correctable. We show that this criterion has a thermodynamical interpretation.

quant-ph↗