arXiv subjects

Yiannis Vlassopoulos

Publications and source records attributed to Yiannis Vlassopoulos.

9 recordsLinked to original sources

arXiv2025

Relu and softplus neural nets as zero-sum turn-based games

We show that the output of a ReLU neural network can be interpreted as the value of a zero-sum, turn-based, stopping game, which we call the ReLU net game. The game runs in the direction opposite to that of the network, and the input of the network serves as the terminal reward of the game. In fact, evaluating the network is the same as running the Shapley-Bellman backward recursion for the value of the game. Using the expression of the value of the game as an expected total payoff with respect to the path measure induced by the transition probabilities and a pair of optimal policies, we derive a discrete Feynman-Kac-type path-integral formula for the network output. This game-theoretic representation can be used to derive bounds on the output from bounds on the input, leveraging the monotonicity of Shapley operators, and to verify robustness properties using policies as certificates. Moreover, training the neural network becomes an inverse game problem: given pairs of terminal rewards and corresponding values, one seeks transition probabilities and rewards of a game that reproduces them. Finally, we show that a similar approach applies to neural networks with Softplus activation functions, where the ReLU net game is replaced by its entropic regularization.

cs.LG↗

arXiv2024

Pre-Calabi-Yau algebras and topological quantum field theories

We introduce a notion generalizing Calabi-Yau structures on A-infinity algebras and categories, which we call pre-Calabi-Yau structures. This notion does not need either one of the finiteness conditions (smoothness or compactness) which are required for Calabi-Yau structures to exist. In terms of noncommutative geometry, a pre-CY structure is as a polyvector field satisfying an integrability condition with respect to a noncommutative analogue of the Schouten-Nijenhuis bracket. We show that a pre-CY structure defines an action of a certain PROP of chains on decorated Riemann surfaces. In the language of the cobordism perspective on TQFTs, this gives a partially defined extended 2-dimensional TQFT, whose 2-dimensional cobordisms are generated only by handles of index one. We present some examples of pre-CY structures appearing naturally in geometric and topological contexts.

math.AG↗

arXiv2024

Smooth Calabi-Yau structures and the noncommutative Legendre transform

We elucidate the relation between smooth Calabi-Yau structures and pre-Calabi-Yau structures. We show that, from a smooth Calabi-Yau structure on an $A_\infty$-category $A$, one can produce a pre-Calabi-Yau structure on $A$; as defined in our previous work, this is a shifted noncommutative version of an integrable polyvector field. We explain how this relation is an analogue of the Legendre transform, and how it defines a one-to-one mapping, in a certain homological sense. For concreteness, we apply this formalism to chains on based loop spaces of (possibly non-simply connected) Poincaré duality spaces, and fully calculate the case of the circle.

math.AT↗

arXiv2024

Directed Metric Structures arising in Large Language Models

Large Language Models are transformer neural networks which are trained to produce a probability distribution on the possible next words to given texts in a corpus, in such a way that the most likely word predicted is the actual word in the training text. In this paper we find what is the mathematical structure defined by such conditional probability distributions of text extensions. Changing the view point from probabilities to -log probabilities we observe that the subtext order is completely encoded in a metric structure defined on the space of texts $\mathcal{L}$, by -log probabilities. We then construct a metric polyhedron $P(\mathcal{L})$ and an isometric embedding (called Yoneda embedding) of $\mathcal{L}$ into $P(\mathcal{L})$ such that texts map to generators of certain special extremal rays. We explain that $P(\mathcal{L})$ is a $(\min,+)$ (tropical) linear span of these extremal ray generators. The generators also satisfy a system of $(\min+)$ linear equations. We then show that $P(\mathcal{L})$ is compatible with adding more text and from this we derive an approximation of a text vector as a Boltzmann weighted linear combination of the vectors for words in that text. We then prove a duality theorem showing that texts extensions and text restrictions give isometric polyhedra (even though they look a priory very different). Moreover we prove that $P(\mathcal{L})$ is the lattice closure of (a version of) the so called, Isbell completion of $\mathcal{L}$ which turns out to be the $(\max,+)$ span of the text extremal ray generators. All constructions have interpretations in category theory but we don't use category theory explicitly. The categorical interpretations are briefly explained in an appendix. In the final appendix we describe how the syntax to semantics problem could fit in a general well known mathematical duality.

cs.LG↗

arXiv2021

Language Modeling with Reduced Densities

This work originates from the observation that today's state-of-the-art statistical language models are impressive not only for their performance, but also - and quite crucially - because they are built entirely from correlations in unstructured text data. The latter observation prompts a fundamental question that lies at the heart of this paper: What mathematical structure exists in unstructured text data? We put forth enriched category theory as a natural answer. We show that sequences of symbols from a finite alphabet, such as those found in a corpus of text, form a category enriched over probabilities. We then address a second fundamental question: How can this information be stored and modeled in a way that preserves the categorical structure? We answer this by constructing a functor from our enriched category of text to a particular enriched category of reduced density operators. The latter leverages the Loewner order on positive semidefinite operators, which can further be interpreted as a toy example of entailment.

cs.CL↗

arXiv2021

An enriched category theory of language: from syntax to semantics

State of the art language models return a natural language text continuation from any piece of input text. This ability to generate coherent text extensions implies significant sophistication, including a knowledge of grammar and semantics. In this paper, we propose a mathematical framework for passing from probability distributions on extensions of given texts, such as the ones learned by today's large language models, to an enriched category containing semantic information. Roughly speaking, we model probability distributions on texts as a category enriched over the unit interval. Objects of this category are expressions in language, and hom objects are conditional probabilities that one expression is an extension of another. This category is syntactical -- it describes what goes with what. Then, via the Yoneda embedding, we pass to the enriched category of unit interval-valued copresheaves on this syntactical category. This category of enriched copresheaves is semantic -- it is where we find meaning, logical operations such as entailment, and the building blocks for more elaborate semantic concepts.

math.CT↗

arXiv2017

Language as a matrix product state

We propose a statistical model for natural language that begins by considering language as a monoid, then representing it in complex matrices with a compatible translation invariant probability measure. We interpret the probability measure as arising via the Born rule from a translation invariant matrix product state.

cs.CL↗

arXiv2017

Tensor network language model

We propose a new statistical model suitable for machine learning of systems with long distance correlations such as natural languages. The model is based on directed acyclic graph decorated by multi-linear tensor maps in the vertices and vector spaces in the edges, called tensor network. Such tensor networks have been previously employed for effective numerical computation of the renormalization group flow on the space of effective quantum field theories and lattice models of statistical mechanics. We provide explicit algebro-geometric analysis of the parameter moduli space for tree graphs, discuss model properties and applications such as statistical translation.

cs.CL↗

arXiv2002

Quantum Cohomology and Morse Theory on the Loop Space of Toric Varieties

On a symplectic manifold $M$, the quantum product defines a complex, one parameter family of flat connections called the A-model or Dubrovin connections. Let $\hbar$ denote the parameter. Associated to them is the quantum $\mathcal{D}$ - module ${\mathcal{D}}/I$ over the Heisenberg algebra of first order differential operators on a complex torus. An element of $I$ gives a relation in the quantum cohomology of $M$ by taking the limit as $\hbar\to 0$. Givental (HomGeom), discovered that there should be a structure of a $\mathcal{D}$ - module on the (as yet not rigorously defined) ${S^1}$ equivariant Floer cohomology of the loop space of $M$ and conjectured that the two modules should be equal. Based on that, we formulate a conjecture about how to compute the quantum cohomology $\mathcal{D}$ - module in terms of Morse theoretic data for the symplectic action functional. The conjecture is proven in the case of toric manifolds with $\int_d{c_1}> 0$ for all nonzero classes $d$ of rational curves in $M$.

math.AG↗