Searcharxiv⌕ Search

arXiv subjects

Eloy Mosig García

Publications and source records attributed to Eloy Mosig García.

3 recordsLinked to original sources

Quantitative Gaussian-Process limits of Tensor Programs

We study the infinite-width Gaussian-process limit of random neural networks through the lens of tensor programs, and we provide a quantitative convergence theory in Wasserstein distance. Our main result gives explicit finite-width error bounds, of order inverse square-root of the widths between finite-network executions and their Gaussian-process limits. The framework is architecture-agnostic and covers feed-forward models together with weight-sharing schemes relevant for recurrent and transformer-type architectures.

cs.LG↗

Stochastic Scaling Limits and Synchronization by Noise in Deep Transformer Models

We prove pathwise convergence of the layerwise evolution of tokens in a finite-depth, finite-width transformer model with MultiLayer Perceptron (MLP) blocks to a continuous-time stochastic interacting particle system. We also identify the stochastic partial differential equation describing the evolution of the tokens' distribution in this limit and prove propagation of chaos when the number of such tokens is large. The bounds we establish are quantitative and the limits we consider commute. We further prove that the limiting stochastic model displays synchronization by noise and establish exponential dissipation of the interaction energy on average, provided that the common noise is sufficiently coercive relative to the deterministic self-attention drift. We finally characterize the activation functions satisfying the former condition.

math.PR↗

The Convex Matching Distance in Multiparameter Persistence

We introduce the convex matching distance, a novel metric for comparing functions with values in the real plane. This metric measures the maximal bottleneck distance between the persistence diagrams associated with the convex combinations of the two function components. Similarly to the traditional matching distance, the convex matching distance aggregates the information provided by two real-valued components. However, whereas the matching distance depends on two parameters, the convex matching distance depends on only one, offering improved computational efficiency. We further show that the convex matching distance can be more discriminative than the traditional matching distance in certain cases, although the two metrics are generally not comparable. Moreover, we prove that the convex matching distance is stable and characterize the coefficients of the convex combination at which it is attained. Finally, we demonstrate that this new aggregation framework benefits from the computational advantages provided by the Pareto grid, a collection of curves in the plane whose points lie in the image of the Pareto critical set associated with functions assuming values on the real plane. Experimental validation on MNIST digits, synthetic shapes, and chaotic attractors suggests that the convex matching distance provides a reliable and efficient alternative to the matching distance, at a significantly lower computational cost.

math.AT↗