Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2503.15421

Probing the topology of the space of tokens with structured prompts

Michael Robinson·Sourya Dey·Taisa Kushner
View original source ↗Read source document ↗Export metadata ↓

Abstract

This article presents a general and flexible method for prompting a large language model (LLM) to reveal its (hidden) token input embedding up to homeomorphism. Moreover, this article provides strong theoretical justification -- a mathematical proof for generic LLMs -- for why this method should be expected to work. With this method in hand, we demonstrate its effectiveness by recovering the token subspace of Llemma-7B. The results of this paper apply not only to LLMs but also to general nonlinear autoregressive processes.

Explore related subjects

math.DGcs.AI

Keep this discovery

BibTeXRIS

Michael Robinson, Sourya Dey, Taisa Kushner. 2025-03-19. Probing the topology of the space of tokens with structured prompts. https://arxiv.org/abs/2503.15421

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

The $q$-deformed cross-ratio: modular invariants and Coxeter friezes

We introduce and study a scalar $q$-deformation of the cross-ratio on $\mathbb P^1(\mathbb Q)$. Our construction is based on the notion of $q$-deformed rational numbers due to Morier-Genoud and the author. The $q$-cross-ratio is invariant under $\mathrm{PSL}(2,\mathbb{Z})$, while elements of determinant $-1$ of $\mathrm{PGL}(2,\mathbb{Z})$ act by $q\mapsto q^{-1}$. A principal result is its relation to $q$-deformed Coxeter friezes associated with rational polygons. The expansion at $q=e^h$ yields an algebraically independent sequence of modular invariants and relative invariants, although this sequence does not separate modular orbits. We compute the first two nonconstant coefficients of this expansion explicitly.

math.DG↗
arXiv2026

The Cartan-Hadamard conjecture in dimension five

We show that the sharp Euclidean isoperimetric inequality holds for domains in complete simply connected Riemannian $5$-manifolds of nonpositive sectional curvature, which establishes the Cartan-Hadamard conjecture in that dimension. The main step is a sharp inequality for constant-mean-curvature hypersurfaces, proved via integrals over pairs of boundary points, in the spirit of Banchoff-Pohl, together with an estimate for Jacobi fields along geodesic chords. The inequality persists for boundaries of isoperimetric regions in geodesic balls, whose mean curvature is constant only on the free part. An isoperimetric-profile argument, after Kleiner, completes the proof. Our method also gives a new proof in dimension $3$.

math.DG↗
arXiv2026

On static manifolds with boundary admitting a nowhere-vanishing static potential

We study complete static manifolds with boundary admitting a nowhere-vanishing static potential. Our main result shows that, under a natural lower bound relating the scalar curvature and the boundary mean curvature, a simple static manifold with boundary must in fact have positive scalar curvature, negative boundary mean curvature, and be compact; we also obtain explicit relations and estimates involving the volume of the manifold and the geometry of its boundary. In the scalar-flat case, we prove global splitting and Ricci-flat rigidity results, including for disconnected boundary, while in the negative scalar curvature case we establish a sharp mean-curvature bound and characterize the equality case by an exponential warped-product structure. The proofs rely essentially on the study of the associated Einstein manifold. In appendix we derive several identities for static manifolds with boundary and discuss the associated Einstein manifold technique in the boundaryless setting.

math.DG↗