arXiv subjects

Bilge Senturk

Publications and source records attributed to Bilge Senturk.

3 recordsLinked to original sources

arXiv2026

Attention Meets Reachability: Structural Equivalence and Efficiency in Grammar-Constrained LLM Decoding

We study grammar-constrained decoding (GCD) as a coupling between an autoregressive next-token distribution and a reachability oracle over a pushdown system compiled from a context-free grammar (CFG). We prove an oracle invariance theorem: language-equivalent grammars induce identical admissible next-token sets for every prefix, hence identical logit masks, yet can yield provably different compiled state spaces and online ambiguity costs. We give exact control-state blowup counts for the canonical $a^n b^n$ language under redundant nonterminal delegation, and introduce a left-to-right structural ambiguity cost (SAC) measuring incremental packed-parse-forest growth per token. For two equivalent grammars over all finite strings, SAC is $O(1)$ per token under right-recursion but $\Theta(t^2)$ per token and $\Theta(n^3)$ cumulatively under concatenation. We establish engine-independent lower bounds: any sound, retrieval-efficient, parse-preserving online masking engine must incur $\Omega(t^2)$ work per token on a specific constant-size CFG family, unconditionally within this model. We define decoding-cost equivalence classes of grammars and prove existence of minimal-SAC representatives within bounded rewrite families. Finally, we characterize the true conditional sampler via a Doob $h$-transform and derive sharp one-step KL and total-variation distortion bounds for hard-masked decoding in terms of survival-probability spread among admissible next tokens. We integrate these results with Transformer and Mixture-of-Experts architectures, derive latency envelopes in terms of vocabulary size, active state sets, and beam width, and connect SAC to instrumentation-based predictive performance models and automated grammar optimization.

cs.CL↗

arXiv2026

Topological Relational Theory: A Simplicial-Complex View of Functional Dependencies, Lossless Decomposition, and Acyclicity

We develop a topological lens on relational schema design by encoding functional dependencies (FDs) as simplices of an abstract simplicial complex. This dependency complex exposes multi-attribute interactions and enables homological invariants (Betti numbers) to diagnose cyclic dependency structure. We define Simplicial Normal Form (SNF) as homological acyclicity of the dependency complex in positive dimensions, i.e., vanishing reduced homology for all $n \ge 1$. SNF is intentionally weaker than contractibility and does not identify homology with homotopy. For decompositions, we give a topological reformulation of the classical binary lossless-join criterion: assuming dependency preservation, a decomposition is lossless exactly when the intersection attributes form a key for at least one component. Topologically, this yields a strong deformation retraction that trivializes the relevant Mayer--Vietoris boundary map. For multiway decompositions, we show how the nerve of a cover by induced subcomplexes provides a computable certificate: a 1-cycle in the nerve (detected by $H_1$) obstructs join-tree structure and aligns with cyclic join behavior in acyclic-scheme theory. Finally, we discuss an algorithmic consequence: Betti numbers of the dependency complex (or of a decomposition nerve) can be computed from boundary matrices and used as a lightweight schema diagnostic to localize "unexplained" dependency cycles, complementing standard FD-chase tests.

cs.DB↗

arXiv2026

The Geometry of Thought: Disclosing the Transformer as a Tropical Polynomial Circuit

We prove that the Transformer self-attention mechanism in the high-confidence regime ($\beta \to \infty$, where $\beta$ is an inverse temperature) operates in the tropical semiring (max-plus algebra). In particular, we show that taking the tropical limit of the softmax attention converts it into a tropical matrix product. This reveals that the Transformer's forward pass is effectively executing a dynamic programming recurrence (specifically, a Bellman-Ford path-finding update) on a latent graph defined by token similarities. Our theoretical result provides a new geometric perspective for chain-of-thought reasoning: it emerges from an inherent shortest-path (or longest-path) algorithm being carried out within the network's computation.

cs.LG↗