arXiv subjects

Theodore Papamarkou

Publications and source records attributed to Theodore Papamarkou.

At least 19 recordsLinked to original sources

arXiv2026

Colored Markov polycategories and diagrammatic differentiation

Many stochastic systems are built by wiring typed components together, but the wiring is often neither purely sequential nor type-homogeneous. This paper develops categorical semantics for such systems using ordered polycategories whose morphisms are Markov kernels. The basic operation is kernel slotwise composition, which connects one output slot of a many-output kernel to one input slot of another and marginalizes the internal wire. We prove its structural laws by assigning trace semantics to finite acyclic diagrams. We then introduce colored Markov polycategories, where objects and kernels carry colors and typed connections are realized by interface kernels satisfying identity and composition laws. This gives a colored kernel slotwise composition and trace semantics for typed stochastic diagrams. To describe systems whose structure changes, we co-index colored Markov polycategories and parameter spaces over an indexing category. Finally, for finite acyclic parameterized diagrams, we prove a diagrammatic differentiation result. The derivative of an expected scalar objective is obtained from local reverse-mode contributions at the parameterized vertices, with stochastic and deterministic kernels handled through local gradient representing functions. The construction gives a typed, compositional language for finite acyclic stochastic systems and their parameter sensitivities.

math.CT↗

arXiv2026

Bayesian control for coding agents

Modern coding agents pair LLM generators with various tools, including cheap diagnostics and expensive verifiers. The tool-use decisions are typically governed by orchestrators that often use fixed rules and ignore uncertainty. We formulate orchestration as cost-sensitive sequential hypothesis testing: a Bayesian controller maintains a belief over candidate correctness and dynamically decides whether to gather more evidence, refine the candidate, verify it, or stop. Across six generators and nine coding benchmarks, Bayesian control proves to be most valuable when verification is costly and critics are informative but imperfect. Beyond control, the belief state yields an interpretable correctness score that outperforms token-probability and raw tool-success baselines for uncertainty quantification.

cs.AI↗

arXiv2026

Coordinate-wise splitting algorithms for ODE simulation via Koopman-Lie product formulas

We present a computational framework for simulating finite-dimensional ordinary differential equations by combining classical Koopman-Lie product formulas with coordinate-wise frozen subflows. The setting is model-known, since the vector field is assumed to be available, and no data-driven approximation of the Koopman operator is attempted. Under standard assumptions, the Koopman-Lie generator associated with the flow admits a coordinate decomposition into partial generators. This decomposition leads to elementary updates in which all but one state variable are frozen, and the resulting frozen scalar subproblems are evaluated either in closed form or by one-dimensional solves. Lie-Trotter, Strang, and higher-order exponential compositions are then converted into state-update algorithms for two- and three-dimensional systems, with the semigroup and product-formula theory used as background justification for the constructions. We also record the exponential-term counts produced by the recursive constructions used in the implementation. These counts are presented as implementation costs. Numerical experiments on the Lotka-Volterra, Van der Pol, and Lorenz systems compare the coordinate-wise splitting algorithms with high-accuracy RK45 reference solutions using root-mean-square errors and work-precision curves. The results illustrate the practical trade-off between splitting order, number of time steps, number of exponential factors, and runtime.

math.NA↗

arXiv2026

A Typed Tensor Language for Federated Learning

Federated learning and analytics are often described as collections of separate protocols, even when they share the same mathematical form: client-local tensor computation, mergeable aggregation into shared state, and shared-only post-processing. We introduce a typed tensor language that formalizes this structure. The language distinguishes federated tensors, whose records are partitioned across clients along a tracked record axis, from shared tensors, which are available globally. Its semantics are defined by comparison with a virtual global tensor, used only as a reference object. The main result is a shared-state factorization theory. We show that typed one-round programs factor through fixed-dimensional shared state whose size is independent of the number of clients and records, computed from client-local tensor expressions and merged across clients. We also prove a converse representability result; factorizations whose encoders and decoders are expressible in the language are realized by typed one-round programs, and the correspondence extends to iterative programs whose cross-round state is shared. This gives a formal account of the computations in the language that can be expressed as encode, merge, and decode procedures. We then develop a differentiable fragment for learning. If a per-record loss and its per-record gradient are represented by client-local tensor expressions, the global gradient is represented by record-axis summation of the federated gradient tensor. This yields typed iterative programs for server-side gradient descent and shared-linear-algebra second-order updates. The framework characterizes a broad class of federated learning computations whose communication passes through fixed-dimensional shared state.

cs.LG↗

arXiv2026

TopoU-Net: a U-Net architecture for topological domains

Many modern datasets mix points, edges, regions, groups, objects, events, hyperedges, and relations. Yet neural architectures often force such data into grids, graphs, or sequences, obscuring higher-order structure and making encoder-decoder designs domain-specific. We view U-Net not as a grid-specific architecture, but as a hierarchical encoder-decoder principle: representation spaces, transport maps between levels, and skip connections between matched levels. Combinatorial complexes naturally supply these ingredients through cells, incidences, and ranks. We introduce TopoU-Net, a rank-path U-Net for topological domains. Given a path from an input rank to a bottleneck rank and back, the encoder lifts cochains upward along incidence maps, the decoder transports them downward, and skip connections merge features at matched ranks. Rank replaces spatial scale: choosing paths through nodes, edges, faces, hyperedges, or global cells becomes the central architectural decision. A key quantity is the bottleneck support ratio, the number of cells at the bottleneck relative to the number of cells at the input rank. This ratio is fixed by the complex and chosen path rather than by arbitrary pooling, and it clarifies when skip connections are optional, useful, or structurally important. Across node classification, graph classification, hypergraph node classification, mesh classification, and image reconstruction, TopoU-Net provides a reusable encoder-decoder template for higher-order structured data. Among the evaluated baselines, it achieves the strongest mean accuracy on six of eight node-classification datasets and four of five hypergraph datasets, with the largest gains on heterophilic graphs. Ablations show that removing skip connections is most damaging under severe bottleneck compression.

cs.LG↗

arXiv2026

Position: agentic AI orchestration should be Bayes-consistent

LLMs excel at predictive tasks and complex reasoning tasks, but many high-value deployments rely on decisions under uncertainty, for example, which tool to call, which expert to consult, or how many resources to invest. While the usefulness and feasibility of Bayesian approaches remain unclear for LLM inference, this position paper argues that the control layer of an agentic AI system (that orchestrates LLMs and tools) is a clear case where Bayesian principles should shine. Bayesian decision theory provides a framework for agentic systems that can help to maintain beliefs over task-relevant latent quantities, to update these beliefs from observed agentic and human-AI interactions, and to choose actions. Making LLMs themselves explicitly Bayesian belief-updating engines remains computationally intensive and conceptually nontrivial as a general modeling target. In contrast, this paper argues that coherent decision-making requires Bayesian principles at the orchestration level of the agentic system, not necessarily the LLM agent parameters. This paper articulates practical properties for Bayesian control that fit modern agentic AI systems and human-AI collaboration, and provides concrete examples and design patterns to illustrate how calibrated beliefs and utility-aware policies can improve agentic AI orchestration.

cs.AI↗

arXiv2026

The Inverse Problem for Single Trajectories of Rough Differential Equations

Motivated by the need to develop a general framework for performing statistical inference for discretely observed random rough differential equations, our aim is to construct a geometric $p$-rough path ${\bf X}$ whose response $Y$, when driving a rough differential equation, matches the observed trajectory $y$. We call this the \textit{continuous inverse problem} and start by rigorously defining its solution. We then develop a framework where the solution can be constructed as a limit of solutions to appropriately designed \textit{discrete inverse problems}, so that convergence holds in $p$-variation. Our approach is based on calibrating the bounded variation paths whose limit defines the rough path `lift' of path $X$ to rough path ${\bf X}$ to the observed trajectory $y$. Moreover, we develop a general numerical algorithm for constructing the solution to the discrete inverse problem. The core idea of the algorithm is to use the signature representation of the path, iterating between the response and the control, each time correcting according to the required properties. We apply our framework to the case where the geometric $p$-rough path ${\bf X}$ is defined as the limit of piecewise linear paths in the $p$-variation topology. We express the discrete inverse problem for a fixed observation rate as a solution to a system of equations driven by piecewise linear paths and prove convergence to the solution of the continuous inverse problem for observation time $δ\to 0$. Finally, we show that, in this context, the numerical algorithm for solving the discrete inverse problem simplifies to an iterative simultaneous update of the local gradients and we prove that it converges in $p$-variation uniformly with respect to $δ$.

math.CA↗

arXiv2026

The Human Brain as a Combinatorial Complex

We propose a framework for constructing combinatorial complexes (CCs) from fMRI time series data that captures both pairwise and higher-order neural interactions through information-theoretic measures, bridging topological deep learning and network neuroscience. Current graph-based representations of brain networks systematically miss the higher-order dependencies that characterize neural complexity, where information processing often involves synergistic interactions that cannot be decomposed into pairwise relationships. Unlike topological lifting approaches that map relational structures into higher-order domains, our method directly constructs CCs from statistical dependencies in the data. Our CCs generalize graphs by incorporating higher-order cells that represent collective dependencies among brain regions, naturally accommodating the multi-scale, hierarchical nature of neural processing. The framework constructs data-driven combinatorial complexes using O-information and S-information measures computed from fMRI signals, preserving both pairwise connections and higher-order cells (e.g., triplets, quadruplets) based on synergistic dependencies. Using NetSim simulations as a controlled proof-of-concept dataset, we demonstrate our CC construction pipeline and show how both pairwise and higher-order dependencies in neural time series can be quantified and represented within a unified structure. This work provides a framework for brain network representation that preserves fundamental higher-order structure invisible to traditional graph methods, and enables the application of topological deep learning (TDL) architectures to neural data.

q-bio.NC↗

arXiv2025

Copresheaf Topological Neural Networks: A Generalized Deep Learning Framework

We introduce copresheaf topological neural networks (CTNNs), a powerful unifying framework that encapsulates a wide spectrum of deep learning architectures, designed to operate on structured data, including images, point clouds, graphs, meshes, and topological manifolds. While deep learning has profoundly impacted domains ranging from digital assistants to autonomous systems, the principled design of neural architectures tailored to specific tasks and data types remains one of the field's most persistent open challenges. CTNNs address this gap by formulating model design in the language of copresheaves, a concept from algebraic topology that generalizes most practical deep learning models in use today. This abstract yet constructive formulation yields a rich design space from which theoretically sound and practically effective solutions can be derived to tackle core challenges in representation learning, such as long-range dependencies, oversmoothing, heterophily, and non-Euclidean domains. Our empirical results on structured data benchmarks demonstrate that CTNNs consistently outperform conventional baselines, particularly in tasks requiring hierarchical or localized sensitivity. These results establish CTNNs as a principled multi-scale foundation for the next generation of deep learning architectures.

cs.LG↗

arXiv2025

Spatial-Aware Decision-Making with Ring Attractors in Reinforcement Learning Systems

Ring attractors, mathematical models inspired by neural circuit dynamics, provide a biologically plausible mechanism to improve learning speed and accuracy in Reinforcement Learning (RL). Serving as specialized brain-inspired structures that encode spatial information and uncertainty, ring attractors explicitly encode the action space, facilitate the organization of neural activity, and enable the distribution of spatial representations across the neural network in the context of Deep Reinforcement Learning (DRL). These structures also provide temporal filtering that stabilizes action selection during exploration, for example, by preserving the continuity between rotation angles in robotic control or adjacency between tactical moves in game-like environments. The application of ring attractors in the action selection process involves mapping actions to specific locations on the ring and decoding the selected action based on neural activity. We investigate the application of ring attractors by both building an exogenous model and integrating them as part of DRL agents. Our approach significantly improves state-of-the-art performance on the Atari 100k benchmark, achieving a 53% increase in performance over selected baselines.

cs.LG↗

arXiv2025

TopoBench: A Framework for Benchmarking Topological Deep Learning

This work introduces TopoBench, an open-source library designed to standardize benchmarking and accelerate research in topological deep learning (TDL). TopoBench decomposes TDL into a sequence of independent modules for data generation, loading, transforming and processing, as well as model training, optimization and evaluation. This modular organization provides flexibility for modifications and facilitates the adaptation and optimization of various TDL pipelines. A key feature of TopoBench is its support for transformations and lifting across topological domains. Mapping the topology and features of a graph to higher-order topological domains, such as simplicial and cell complexes, enables richer data representations and more fine-grained analyses. The applicability of TopoBench is demonstrated by benchmarking several TDL architectures across diverse tasks and datasets.

cs.LG↗

arXiv2024

TopoX: A Suite of Python Packages for Machine Learning on Topological Domains

We introduce TopoX, a Python software suite that provides reliable and user-friendly building blocks for computing and machine learning on topological domains that extend graphs: hypergraphs, simplicial, cellular, path and combinatorial complexes. TopoX consists of three packages: TopoNetX facilitates constructing and computing on these domains, including working with nodes, edges and higher-order cells; TopoEmbedX provides methods to embed topological domains into vector spaces, akin to popular graph-based embedding algorithms such as node2vec; TopoModelX is built on top of PyTorch and offers a comprehensive toolbox of higher-order message passing functions for neural networks on topological domains. The extensively documented and unit-tested source code of TopoX is available under MIT license at https://pyt-team.github.io/}{https://pyt-team.github.io/.

cs.LG↗

arXiv2024

ICML Topological Deep Learning Challenge 2024: Beyond the Graph Domain

This paper describes the 2nd edition of the ICML Topological Deep Learning Challenge that was hosted within the ICML 2024 ELLIS Workshop on Geometry-grounded Representation Learning and Generative Modeling (GRaM). The challenge focused on the problem of representing data in different discrete topological domains in order to bridge the gap between Topological Deep Learning (TDL) and other types of structured datasets (e.g. point clouds, graphs). Specifically, participants were asked to design and implement topological liftings, i.e. mappings between different data structures and topological domains --like hypergraphs, or simplicial/cell/combinatorial complexes. The challenge received 52 submissions satisfying all the requirements. This paper introduces the main scope of the challenge, and summarizes the main results and findings.

cs.LG↗

arXiv2024

Model-agnostic variable importance for predictive uncertainty: an entropy-based approach

In order to trust the predictions of a machine learning algorithm, it is necessary to understand the factors that contribute to those predictions. In the case of probabilistic and uncertainty-aware models, it is necessary to understand not only the reasons for the predictions themselves, but also the reasons for the model's level of confidence in those predictions. In this paper, we show how existing methods in explainability can be extended to uncertainty-aware models and how such extensions can be used to understand the sources of uncertainty in a model's predictive distribution. In particular, by adapting permutation feature importance, partial dependence plots, and individual conditional expectation plots, we demonstrate that novel insights into model behaviour may be obtained and that these methods can be used to measure the impact of features on both the entropy of the predictive distribution and the log-likelihood of the ground truth labels under that distribution. With experiments using both synthetic and real-world data, we demonstrate the utility of these approaches to understand both the sources of uncertainty and their impact on model performance.

stat.ML↗

arXiv2024

Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI

In the current landscape of deep learning research, there is a predominant emphasis on achieving high predictive accuracy in supervised tasks involving large image and language datasets. However, a broader perspective reveals a multitude of overlooked metrics, tasks, and data types, such as uncertainty, active and continual learning, and scientific data, that demand attention. Bayesian deep learning (BDL) constitutes a promising avenue, offering advantages across these diverse settings. This paper posits that BDL can elevate the capabilities of deep learning. It revisits the strengths of BDL, acknowledges existing challenges, and highlights some exciting research avenues aimed at addressing these obstacles. Looking ahead, the discussion focuses on possible ways to combine large-scale foundation models with BDL to unlock their full potential.

cs.LG↗

arXiv2024

Position: Topological Deep Learning is the New Frontier for Relational Learning

Topological deep learning (TDL) is a rapidly evolving field that uses topological features to understand and design deep learning models. This paper posits that TDL is the new frontier for relational learning. TDL may complement graph representation learning and geometric deep learning by incorporating topological concepts, and can thus provide a natural choice for various machine learning settings. To this end, this paper discusses open problems in TDL, ranging from practical benefits to theoretical foundations. For each problem, it outlines potential solutions and future research opportunities. At the same time, this paper serves as an invitation to the scientific community to actively participate in TDL research to unlock the potential of this emerging field.

cs.LG↗

arXiv2024

Connecting the Dots: Is Mode-Connectedness the Key to Feasible Sample-Based Inference in Bayesian Neural Networks?

A major challenge in sample-based inference (SBI) for Bayesian neural networks is the size and structure of the networks' parameter space. Our work shows that successful SBI is possible by embracing the characteristic relationship between weight and function space, uncovering a systematic link between overparameterization and the difficulty of the sampling problem. Through extensive experiments, we establish practical guidelines for sampling and convergence diagnosis. As a result, we present a deep ensemble initialized approach as an effective solution with competitive performance and uncertainty quantification.

cs.LG↗

arXiv2024

Probability-Generating Function Kernels for Spherical Data

Probability-generating function (PGF) kernels are introduced, which constitute a class of kernels supported on the unit hypersphere, for the purposes of spherical data analysis. PGF kernels generalize RBF kernels in the context of spherical data. The properties of PGF kernels are studied. A semi-parametric learning algorithm is introduced to enable the use of PGF kernels with spherical data.

stat.ML↗