arXiv subjects

Cameron Freer

Publications and source records attributed to Cameron Freer.

At least 19 recordsLinked to original sources

arXiv2026

Forcing with Invariant Measures

This paper introduces a model-theoretic generalization of the notion of forcing with random reals, in which forcing gives rise to random generic structures. Specifically, we consider forcing with $\kappa$-Borel probability measures on the space of $L$-structures with a (possibly uncountable) infinite set $X$, focusing on those that are invariant under the action of the symmetric group $Sym(X)$. We demonstrate how any $Sym(X)$-invariant measure where $X$ is countable can be uniquely extended to a $Sym(Y)$-invariant measure where $Y$ is uncountable, and prove that forcing with such measures satisfies the countable chain condition. We also show that we can uniformly distinguish between these random generic structures and the Cohen generic structures that arise from forcing with a strong Fra\"iss\'e class: There is a $\kappa$-Borel set of low complexity that contains every Cohen generic structure that is not highly homogeneous but contains no random generic structure, implying that a structure that is not highly homogeneous cannot be both Cohen generic and random generic. Finally, we answer an open question of Kostana in the case of $\omega_1$, by establishing a connection between forcing with a strong Fra\"iss\'e class and Cohen forcing.

math.LO↗

arXiv2026

Faster LLM Inference via Sequential Monte Carlo

Speculative decoding (SD) accelerates language model inference by drafting tokens from a cheap proposal model and verifying them against an expensive target model via rejection sampling. Because rejection truncates the draft block at the first error, throughput degrades when draft and target diverge. Rather than rejecting draft tokens outright, we propose to reweight them. To this end, we introduce sequential Monte Carlo speculative decoding (SMC-SD), which replaces token-level rejection with importance-weighted resampling over a population of draft particles. SMC-SD is a principled approximate inference scheme that trades exactness for additional speed, while preserving theoretical bounds on its per-step approximation error. Because LLM inference is memory bandwidth-bound, the arithmetic needed to draft particles and to score them in parallel comes nearly for free -- SMC-SD uses idle compute to turn verification into a vectorized, fixed-size operation with no rollback. Empirically, SMC-SD achieves 2.36x speed-up over speculative decoding and a 5.2x speed-up over autoregressive decoding, while remaining within 3% of the target model's accuracy on reasoning, instruction-following, and coding benchmarks.

cs.LG↗

arXiv2026

On the computability of cofinal Fra\"iss\'e limits

For any collection of finite structures closed under isomorphism (i.e., an age) which has the Hereditary Property (HP), the Joint Embedding Property (JEP), and the Cofinal Amalgamation Property (CAP), there is a unique (up to isomorphism) countable structure which is cofinally ultrahomogeneous with the given age. Such a structure is called the cofinal Fra\"iss\'e limit of the age. In this paper, we consider the computational strength needed to construct the cofinal Fra\"iss\'e limit of a computable age. We show that this construction can always be done using the oracle 0''', and that there are ages that require 0''. In contrast, we show that if one assumes the strengthening of (CAP) known as the Amalgamation Property (AP), then the resulting limit, called the Fra\"iss\'e limit, can be constructed from the age using 0'. Our results therefore show that the more general case of cofinal Fra\"iss\'e limits requires greater computational strength than Fra\"iss\'e limits.

math.LO↗

arXiv2025

Generic sampling and invariant measures on the space of $k$-uniform hypergraphs

We prove a model-theoretic representation theorem for the distribution of an ergodic exchangeable $k$-uniform hypergraph: every such measure arises as the pushforward of the countably-iterated Morley product of a global Borel-definable Keisler measure over the countable universal homogeneous $k$-uniform hypergraph. We show this by starting with a Borel $k$-hypergraphon $W$ and constructing a Keisler measure $\mu_{W}$ such that generic sampling with respect to $\mu_{W}$ yields the same invariant measure as does the standard hypergraphon sampling procedure with respect to $W$. When $k = 2$, our results give a new representation theorem for ergodic exchangeable graphs via Keisler measures over a monster model of the Rado graph.

math.CO↗

arXiv2024

Data for Mathematical Copilots: Better Ways of Presenting Proofs for Machine Learning

The datasets and benchmarks commonly used to train and evaluate the mathematical capabilities of AI-based mathematical copilots (primarily large language models) exhibit several shortcomings and misdirections. These range from a restricted scope of mathematical complexity to limited fidelity in capturing aspects beyond the final, written proof (e.g. motivating the proof, or representing the thought processes leading to a proof). These issues are compounded by a dynamic reminiscent of Goodhart's law: as benchmark performance becomes the primary target for model development, the benchmarks themselves become less reliable indicators of genuine mathematical capability. We systematically explore these limitations and contend that enhancing the capabilities of large language models, or any forthcoming advancements in AI-based mathematical assistants (copilots or ``thought partners''), necessitates a course correction both in the design of mathematical datasets and the evaluation criteria of the models' mathematical ability. In particular, it is necessary for benchmarks to move beyond the existing result-based datasets that map theorem statements directly to proofs, and instead focus on datasets that translate the richer facets of mathematical research practice into data that LLMs can learn from. This includes benchmarks that supervise the proving process and the proof discovery process itself, and we advocate for mathematical dataset developers to consider the concept of "motivated proof", introduced by G. P\'olya in 1949, which can serve as a blueprint for datasets that offer a better proof learning signal, alleviating some of the mentioned limitations.

cs.LG↗

arXiv2021

On computable learning of continuous features

We introduce definitions of computable PAC learning for binary classification over computable metric spaces. We provide sufficient conditions for learners that are empirical risk minimizers (ERM) to be computable, and bound the strong Weihrauch degree of an ERM learner under more general conditions. We also give a presentation of a hypothesis class that does not admit any proper computable PAC learner with computable sample function, despite the underlying class being PAC learnable.

cs.LG↗

arXiv2021

On computable aspects of algebraic and definable closure

We investigate the computability of algebraic closure and definable closure with respect to a collection of formulas. We show that for a computable collection of formulas of quantifier rank at most $n$, in any given computable structure, both algebraic and definable closure with respect to that collection are $\Sigma^0_{n+2}$ sets. We further show that these bounds are tight.

math.LO↗

arXiv2020

Deep Involutive Generative Models for Neural MCMC

We introduce deep involutive generative models, a new architecture for deep generative modeling, and use them to define Involutive Neural MCMC, a new approach to fast neural MCMC. An involutive generative model represents a probability kernel $G(\phi \mapsto \phi')$ as an involutive (i.e., self-inverting) deterministic function $f(\phi, \pi)$ on an enlarged state space containing auxiliary variables $\pi$. We show how to make these models volume preserving, and how to use deep volume-preserving involutive generative models to make valid Metropolis-Hastings updates based on an auxiliary variable scheme with an easy-to-calculate acceptance ratio. We prove that deep involutive generative models and their volume-preserving special case are universal approximators for probability kernels. This result implies that with enough network capacity and training time, they can be used to learn arbitrarily complex MCMC updates. We define a loss function and optimization algorithm for training parameters given simulated data. We also provide initial experiments showing that Involutive Neural MCMC can efficiently explore multi-modal distributions that are intractable for Hybrid Monte Carlo, and can converge faster than A-NICE-MC, a recently introduced neural MCMC technique.

stat.ML↗

arXiv2018

The entropy function of an invariant measure

Given a countable relational language $L$, we consider probability measures on the space of $L$-structures with underlying set $\mathbb{N}$ that are invariant under the logic action. We study the growth rate of the entropy function of such a measure, defined to be the function sending $n \in \mathbb{N}$ to the entropy of the measure induced by restrictions to $L$-structures on $\{0, \ldots, n-1\}$. When $L$ has finitely many relation symbols, all of arity $k\ge 1$, and the measure has a property called non-redundance, we show that the entropy function is of the form $Cn^k+o(n^k)$, generalizing a result of Aldous and Janson. When $k\ge 2$, we show that there are invariant measures whose entropy functions grow arbitrarily fast in $o(n^k)$, extending a result of Hatami-Norine. For possibly infinite languages $L$, we give an explicit upper bound on the entropy functions of non-redundant invariant measures in terms of the number of relation symbols in $L$ of each arity; this implies that finite-valued entropy functions can grow arbitrarily fast.

math.LO↗

arXiv2017

Stable regularity for relational structures

We generalize the stable graph regularity lemma of Malliaris and Shelah to the case of finite structures in finite relational languages, e.g., finite hypergraphs. We show that under the model-theoretic assumption of stability, such a structure has an equitable regularity partition of size polynomial in the reciprocal of the desired accuracy, and such that for each $k$-ary relation and $k$-tuple of parts of the partition, the density is close to either 0 or 1. In addition, we provide regularity results for finite and Borel structures that satisfy a weaker notion that we call almost stability.

math.LO↗

arXiv2017

Properly ergodic structures

We consider ergodic $\mathrm{Sym}(\mathbb{N})$-invariant probability measures on the space of $L$-structures with domain $\mathbb{N}$ (for $L$ a countable relational language), and call such a measure a properly ergodic structure when no isomorphism class of structures is assigned measure $1$. We characterize those theories in countable fragments of $\mathcal{L}_{\omega_1, \omega}$ for which there is a properly ergodic structure concentrated on the models of the theory. We show that for a countable fragment $F$ of $\mathcal{L}_{\omega_1, \omega}$ the almost-sure $F$-theory of a properly ergodic structure has continuum-many models (an analogue of Vaught's Conjecture in this context), but its full almost-sure $\mathcal{L}_{\omega_1, \omega}$-theory has no models. We also show that, for an $F$-theory $T$, if there is some properly ergodic structure that concentrates on the class of models of $T$, then there are continuum-many such properly ergodic structures.

math.LO↗

arXiv2017

Countable infinitary theories admitting an invariant measure

Let $L$ be a countable language. We characterize, in terms of definable closure, those countable theories $\Sigma$ of $\mathcal{L}_{\omega_1, \omega}(L)$ for which there exists an $S_\infty$-invariant probability measure on the collection of models of $\Sigma$ with underlying set $\mathbb{N}$. Restricting to $\mathcal{L}_{\omega, \omega}(L)$, this answers an open question of Gaifman from 1964, via a translation between $S_\infty$-invariant measures and Gaifman's symmetric measure-models with strict equality. It also extends the known characterization in the case where $\Sigma$ implies a Scott sentence. To establish our result, we introduce machinery for building invariant measures from a directed system of countable structures with measures.

math.LO↗

arXiv2017

On the computability of graph Turing machines

We consider graph Turing machines, a model of parallel computation on a graph, in which each vertex is only capable of performing one of a finite number of operations. This model of computation is a natural generalization of several well-studied notions of computation, including ordinary Turing machines, cellular automata, and parallel graph dynamical systems. We analyze the power of computations that can take place in this model, both in terms of the degrees of computability of the functions that can be computed, and the time and space resources needed to carry out these computations. We further show that properties of the underlying graph have significant consequences for the power of computation thereby obtained. In particular, we show that every arithmetically definable set can be computed by a graph Turing machine in constant time, and that every computably enumerable Turing degree can be computed in constant time and linear space by a graph Turing machine whose underlying graph has finite degree.

math.LO↗

arXiv2015

Priors on exchangeable directed graphs

Directed graphs occur throughout statistical modeling of networks, and exchangeability is a natural assumption when the ordering of vertices does not matter. There is a deep structural theory for exchangeable undirected graphs, which extends to the directed case via measurable objects known as digraphons. Using digraphons, we first show how to construct models for exchangeable directed graphs, including special cases such as tournaments, linear orderings, directed acyclic graphs, and partial orderings. We then show how to construct priors on digraphons via the infinite relational digraphon model (di-IRM), a new Bayesian nonparametric block model for exchangeable directed graphs, and demonstrate inference on synthetic data.

math.ST↗

arXiv2014

A classification of orbits admitting a unique invariant measure

We consider the space of countable structures with fixed underlying set in a given countable language. We show that the number of ergodic probability measures on this space that are $S_\infty$-invariant and concentrated on a single isomorphism class must be zero, or one, or continuum. Further, such an isomorphism class admits a unique $S_\infty$-invariant probability measure precisely when the structure is highly homogeneous; by a result of Peter J. Cameron, these are the structures that are interdefinable with one of the five reducts of the rational linear order $(\mathbb{Q}, <)$.

math.LO↗

arXiv2014

An iterative step-function estimator for graphons

Exchangeable graphs arise via a sampling procedure from measurable functions known as graphons. A natural estimation problem is how well we can recover a graphon given a single graph sampled from it. One general framework for estimating a graphon uses step-functions obtained by partitioning the nodes of the graph according to some clustering algorithm. We propose an iterative step-function estimator (ISFE) that, given an initial partition, iteratively clusters nodes based on their edge densities with respect to the previous iteration's partition. We analyze ISFE and demonstrate its performance in comparison with other graphon estimation techniques.

math.ST↗

arXiv2014

Algorithmic aspects of Lipschitz functions

We characterize the variation functions of computable Lipschitz functions. We show that a real z is computably random if and only if every computable Lipschitz function is differentiable at z. Beyond these principal results, we show that a real z is Schnorr random if and only if every Lipschitz function with $L_1$-computable derivative is differentiable at z.

math.LO↗

arXiv2013

Invariant measures via inverse limits of finite structures

Building on recent results regarding symmetric probabilistic constructions of countable structures, we provide a method for constructing probability measures, concentrated on certain classes of countably infinite structures, that are invariant under all permutations of the underlying set that fix all constants. These measures are constructed from inverse limits of measures on certain finite structures. We use this construction to obtain invariant probability measures concentrated on the classes of countable models of certain first-order theories, including measures that do not assign positive measure to the isomorphism class of any single model. We also characterize those transitive Borel G-spaces admitting a G-invariant probability measure, when G is an arbitrary countable product of symmetric groups on a countable set.

math.LO↗