arXiv subjects

Alejandro Francisco Queiruga

Publications and source records attributed to Alejandro Francisco Queiruga.

3 recordsLinked to original sources

arXiv2026

Interpretability and Generalization Bounds for Learning Spatial Physics

While there are many applications of ML to scientific problems that look promising, visuals can be deceiving. Using numerical analysis techniques, we rigorously quantify the accuracy, convergence rates, and generalization bounds of certain ML models applied to linear differential equations for parameter discovery or solution finding. Beyond the quantity and discretization of data, we identify that the function space of the data is critical to the generalization of the model. A similar lack of generalization is empirically demonstrated for commonly used models, including physics-specific techniques. Counterintuitively, we find that different classes of models can exhibit opposing generalization behaviors. Based on our theoretical analysis, we also introduce a new mechanistic interpretability lens on scientific models whereby Green's function representations can be extracted from the weights of black-box models. Our results inform a new cross-validation technique for measuring generalization in physical systems, which can serve as a benchmark.

cs.LG↗

arXiv2026

Divine Benevolence is an $x^2$: GLUs scale asymptotically faster than MLPs

Scaling laws can be understood from ground-up numerical analysis, where traditional function approximation theory can explain shifts in model architecture choices. GLU variants now dominate frontier LLMs and similar outer-product architectures are prevalent in ranking models. The success of these architectures has mostly been left as an empirical discovery. In this paper, we apply the tools of numerical analysis to expose a key factor: these models have an $x^2$ which enables \emph{asymptotically} faster scaling than MLPs. GLUs have piecewise quadratic functional forms that are sufficient to exhibit quadratic order of approximation. Our key contribution is to demonstrate that the $L(P)$ scaling slope is $L(P)\propto P^{-3}$ for GLUs but only $L(P)=P^{-2}$ for MLPs on function reconstruction problems. We provide a parameter construction and empirical verification of these slopes for 1D function approximation. From the first principles we discover, we make one stride and propose the ``Gated Quadratic Unit'' which has an even steeper $L(P)$ slope than the GLU and MLP. This opens the possibility of architecture design from first principles numerical theory to unlock superior scaling in large models. Replication code is available at https://github.com/afqueiruga/divine_scaling.

cs.LG↗

arXiv2019

Studying Shallow and Deep Convolutional Neural Networks as Learned Numerical Schemes on the 1D Heat Equation and Burgers' Equation

This paper examines the coincidence of neural networks with numerical methods for solving spatiotemporal physical problems. Neural networks are used to learn predictive numerical models from trajectory datasets from two well understood 1D problems: the heat equation and the inviscid Burgers' equation. Coincidence with established numerical methods is shown by demonstrating that a single layer convolutional neural network (CNN) converges to a traditional finite difference stencil for the heat equation. However, a discriminator-based adversarial training method, such as those used in generative adversarial networks (GANs), does not find the expected weights. A compact deep CNN is applied to nonlinear Burgers' equation, where the models' architecture is reminiscent of existing winding finite volume methods. By searching over architectures and using multiple recurrent steps in the training loss, a model is found that can integrate in time, recurring on its outputs, with similar accuracy and stability to Godunov's method.

physics.comp-ph↗