arXiv subjects

Gabriele Santin

Publications and source records attributed to Gabriele Santin.

At least 19 recordsLinked to original sources

arXiv2026

Differentiable Kernel Ridge Regression for Deep Learning Pipelines

Deep neural networks dominate modern machine learning, while alternative function approximators remain comparatively underexplored at scale. In this work, we revisit kernel methods as drop-in components for standard deep learning pipelines. We introduce \emph{Sparse Kernels} (SKs), a differentiable, localized, and lazy variant of kernel ridge regression (KRR) that defers training to inference time and reduces to the solution of small local systems. We integrate SKs into PyTorch as modular layers that preserve end-to-end trainability, and we show that they expose three distinct sets of parameters -- feature representations, target values, and evaluation points -- each of which can be fixed or learned. This decomposition broadens the design space available to practitioners, enabling, in particular, training-free transfer, nonlinear probing, and hybrid kernel-neural models. Across convolutional networks, vision transformers, and reinforcement learning, SK-based modules serve two complementary roles: in some settings, they match the performance of trained neural readouts with substantially less training; in others, they augment existing models and improve their performance when used as additional components. Our results suggest that kernel methods, once made scalable and differentiable, can be readily integrated with deep learning rather than treated as a separate paradigm.

cs.LG↗

arXiv2026

Piecewise linear interpolation via kernels

We consider piecewise linear interpolation from the perspective of kernel interpolation and quadrature. If the Sobolev space $W_2^1(0, 1)$ is equipped with a suitable inner product, its reproducing kernel is piecewise linear and gives rise to piecewise linear interpolation. We show that such kernels are Green kernels for certain second-order partial differential equations and use kernel-based superconvergence theory to obtain rates of convergence for approximation of functions lying in $W_2^s(0, 1)$ for $s \in [1, 2]$. The rates coincide with classical rates for linear splines.

math.NA↗

arXiv2026

On Universality Classes of Equivariant Networks

Equivariant neural networks provide a principled framework for incorporating symmetry into learning architectures and have been extensively analyzed through the lens of their separation power, that is, the ability to distinguish inputs modulo symmetry. This notion plays a central role in settings such as graph learning, where it is often formalized via the Weisfeiler-Leman hierarchy. In contrast, the universality of equivariant models-their capacity to approximate target functions-remains comparatively underexplored. In this work, we investigate the approximation power of equivariant neural networks beyond separation constraints. We show that separation power does not fully capture expressivity: models with identical separation power may differ in their approximation ability. To demonstrate this, we characterize the universality classes of shallow invariant networks, providing a general framework for understanding which functions these architectures can approximate. Since equivariant models reduce to invariant ones under projection, this analysis yields sufficient conditions under which shallow equivariant networks fail to be universal. Conversely, we identify settings where shallow models do achieve separation-constrained universality. These positive results, however, depend critically on structural properties of the symmetry group, such as the existence of adequate normal subgroups, which may not hold in important cases like permutation symmetry.

cs.LG↗

arXiv2026

Refined rates of convergence for target-data dependent greedy generalized interpolation with Sobolev kernels

Greedy methods have recently been successfully applied to generalized kernel interpolation, or the recovery of a function from data stemming from the evaluation of linear functionals, including the approximation of solutions of linear PDEs by symmetric collocation. When applied to kernels generating Sobolev spaces as their native Hilbert spaces, some of these greedy methods can provide the same error guarantee of generalized interpolation on quasi-uniform points. More importantly, certain target-data-adaptive methods even give a dimension- and smoothness-independent improvement in the speed of convergence over quasi-uniform points, thus offering advantages for high-dimensional problems. These convergence rates however contain a spurious logarithmic term that limits this beneficial effect. The goal of this note is to remove this factor, and this is possible by using estimates on metric entropy numbers.

math.NA↗

arXiv2026

On the optimal shape parameter for kernel methods: Sharp direct and inverse statements

The search for the optimal shape parameter for Radial Basis Function (RBF) kernel approximation has been an outstanding research problem for decades. In this work, we establish a theoretical framework for this problem by leveraging a recently established theory on sharp direct, inverse and saturation statements for kernel based approximation. In particular, we link the search for the optimal shape parameter to superconvergence phenomena. Our analysis is carried out for finitely smooth Sobolev kernels, thereby covering large classes of radial kernels used in practice, including those emerging from current machine-learning methodologies. Our results elucidate how approximation regimes, kernel regularity, and parameter choices interact, thereby clarifying a question that has remained unresolved for decades.

math.NA↗

arXiv2025

Weak convergence rates for spectral regularization via sampling inequalities

Convergence rates in spectral regularization methods quantify the approximation error in inverse problems as a function of the noise level or the number of sampling points. Classical strong convergence rate results typically rely on source conditions, which are essential for estimating the truncation error. However, in the framework of kernel approximation, the truncation error in the case of Tikhonov regularization can be characterized entirely through sampling inequalities, without invoking source conditions. In this paper, we first generalize sampling inequalities to spectral regularization, and then, by exploiting the connection between inverse problems and kernel approximation, we derive weak convergence rate bounds for inverse problems, independently of source conditions. These weak convergence rates are established and analyzed when the forward operator is compact and uniformly bounded, or the kernel operator is of trace class.

math.NA↗

arXiv2025

Kernel-based Greedy Approximation of Parametric Elliptic Boundary Value Problems

We recently introduced a scale of kernel-based greedy schemes for approximating the solutions of elliptic boundary value problems. The procedure is based on a generalized interpolation framework in reproducing kernel Hilbert spaces and was coined PDE-$\beta$-greedy procedure, where the parameter $\beta \geq 0$ is used in a greedy selection criterion and steers the degree of function adaptivity. Algebraic convergence rates have been obtained for Sobolev-space kernels and solutions of finite smoothness. We now report a result of exponential convergence rates for the case of infinitely smooth kernels and solutions. We furthermore extend the approximation scheme to the case of parametric PDEs by the use of state-parameter product kernels. In the surrogate modelling context, the resulting approach can be interpreted as an a priori model reduction approach, as no solution snapshots need to be precomputed. Numerical results show the efficiency of the approximation procedure for problems which occur as challenges for other parametric MOR procedures: non-affine geometry parametrizations, moving sources or high-dimensional domains.

math.NA↗

arXiv2025

Forecasting Seasonal Influenza Epidemics with Physics-Informed Neural Networks

Accurate epidemic forecasting is critical for informing public health decisions and timely interventions. While Physics-Informed Neural Networks have shown promise in various scientific domains, their potential application to real-time epidemic forecasting remains underexplored. Here, we present SIR-INN, a hybrid forecasting framework that integrates the mechanistic structure of the classical Susceptible-Infectious-Recovered (SIR) model into a neural network architecture. Trained once on synthetic epidemic scenarios, the model is able to generalize across epidemic conditions without retraining. From limited and noisy observations, SIR-INN infers key transmission parameters via Markov chain Monte Carlo, generating probabilistic short- and long-term forecasts. We validate SIR-INN using national influenza data from the Italian National Institute of Health in the 2023-2024 and 2024-2025 seasons. The model performs competitively with current state-of-the-art approaches, particularly in terms of Weighted Interval Score. It shows accurate predictive performance in nearly all phases of the outbreak, with improved accuracy observed for the 2024-2025 influenza season. Credible uncertainty intervals are consistently maintained, while coverage metrics highlight room for improvement in uncertainty calibration. SIR-INN offers a computationally efficient, transparent, and generalizable solution for epidemic forecasting, appropriately leveraging the framework's hybrid design. Its ability to provide real-time predictions of epidemic dynamics, together with uncertainty quantification, makes it a promising tool for real-world epidemic forecasting.

physics.soc-ph↗

arXiv2025

General superconvergence for kernel-based approximation

Kernel interpolation is a fundamental technique for approximating functions from scattered data, with a well-understood convergence theory when interpolating elements of a reproducing kernel Hilbert space. Beyond this classical setting, research has focused on two regimes: misspecified interpolation, where the kernel smoothness exceeds that of the target function, and superconvergence, where the target is smoother than the Hilbert space. This work addresses the latter, where smoother target functions yield improved convergence rates, and extends existing results by characterizing superconvergence for projections in general Hilbert spaces. We show that functions lying in ranges of certain operators, including adjoint of embeddings, exhibit accelerated convergence, which we extend across interpolation scales between these ranges and the full Hilbert space. In particular, we analyze Mercer operators and embeddings into $L_p$ spaces, linking the images of adjoint operators to Mercer power spaces. Applications to Sobolev spaces are discussed in detail, highlighting how superconvergence depends critically on boundary conditions. Our findings generalize and refine previous results, offering a broader framework for understanding and exploiting superconvergence. The results are supported by numerical experiments.

math.NA↗

arXiv2025

Tuning Butterworth filter's parameters in SPECT reconstructions via kernel-based Bayesian optimization with a no-reference image evaluation metric

In Single Photon Emission Computed Tomography (SPECT), the image reconstruction process involves many tunable parameters that have a significant impact on the quality of the resulting clinical images. Traditional image quality evaluation often relies on expert judgment and full-reference metrics such as MSE and SSIM. However, these approaches are limited by their subjectivity or the need for a ground-truth image. In this paper, we investigate the usage of a no-reference image quality assessment method tailored for SPECT imaging, employing the Perception-based Image QUality Evaluator (PIQUE) score. Precisely, we propose a novel application of PIQUE in evaluating SPECT images reconstructed via filtered backprojection using a parameter-dependent Butterworth filter. For the optimization of filter's parameters, we adopt a kernel-based Bayesian optimization framework grounded in reproducing kernel Hilbert space theory, highlighting the connections to recent greedy approximation techniques. Experimental results in a concrete clinical setting for SPECT imaging show the potential of this optimization approach for an objective and quantitative assessment of image quality, without requiring a reference image.

math.NA↗

arXiv2025

Analysis of Structured Deep Kernel Networks

In this paper, we leverage a recent deep kernel representer theorem to connect kernel based learning and (deep) neural networks in order to understand their interplay. In particular, we show that the use of special types of kernels yields models reminiscent of neural networks that are founded in the same theoretical framework of classical kernel methods, while benefiting from the computational advantages of deep neural networks. Especially the introduced Structured Deep Kernel Networks (SDKNs) can be viewed as neural networks (NNs) with optimizable activation functions obeying a representer theorem. This link allows us to analyze also NNs within the framework of kernel networks. We prove analytic properties of the SDKNs which show their universal approximation properties in three different asymptotic regimes of unbounded number of centers, width and depth. Especially in the case of unbounded depth, more accurate constructions can be achieved using fewer layers compared to corresponding constructions for ReLU neural networks. This is made possible by leveraging properties of kernel approximation.

cs.LG↗

arXiv2025

Adaptive meshfree approximation for linear elliptic partial differential equations with PDE-greedy kernel methods

We consider meshless approximation for solutions of boundary value problems (BVPs) of elliptic Partial Differential Equations (PDEs) via symmetric kernel collocation. We discuss the importance of the choice of the collocation points, in particular by using greedy kernel methods. We introduce a scale of PDE-greedy selection criteria that generalizes existing techniques, such as the PDE-$P$-greedy and the PDE-$f$-greedy rules for collocation point selection. For these greedy selection criteria we provide bounds on the approximation error in terms of the number of greedily selected points and analyze the corresponding convergence rates. This is achieved by a novel analysis of Kolmogorov widths of special sets of BVP point-evaluation functionals. Especially, we prove that target-data dependent algorithms that make use of the right hand side functions of the BVP exhibit faster convergence rates than the target-data independent PDE-$P$-greedy. The convergence rate of the PDE-$f$-greedy possesses a dimension independent rate, which makes it amenable to mitigate the curse of dimensionality. The advantages of these greedy algorithms are highlighted by numerical examples.

math.NA↗

arXiv2025

An algorithm for a constrained P-spline

Regression splines are largely used to investigate and predict data behavior, attracting the interest of mathematicians for their beautiful numerical properties, and of statisticians for their versatility with respect to the applications. Several penalized spline regression models are available in the literature, and the most commonly used ones in real-world applications are P-splines, which enjoy the advantages of penalized models while being easy to generalize across different functional spaces and higher degree order, because of their discrete penalty term. To face the different requirements imposed by the nature of the problem or the physical meaning of the expected values, the P-spline definition is often modified by additional hypotheses, often translated into constraints on the solution or its derivatives. In this framework, our work is motivated by the aim of getting approximation models that fall within pre-established thresholds. Specifically, starting from a set of observed data, we consider a P-spline constrained between some prefixed bounds. In our paper, we just consider 0 as lower bound, although our approach applies to more general cases. We propose to get nonnegativity by imposing lower bounds on selected sample points. The spline can be computed through a sequence of linearly constrained problems. We suggest a strategy to dynamically select the sample points, to avoid extremely dense sampling, and therefore try to reduce as much as possible the computational burden. We show through some computational experiments the reliability of our approach and the accuracy of the results compared to some state-of-the-art models.

stat.ME↗

arXiv2024

Kernel Methods for the Approximation of the Eigenfunctions of the Koopman Operator

The Koopman operator provides a linear framework to study nonlinear dynamical systems. Its spectra offer valuable insights into system dynamics, but the operator can exhibit both discrete and continuous spectra, complicating direct computations. In this paper, we introduce a kernel-based method to construct the principal eigenfunctions of the Koopman operator without explicitly computing the operator itself. These principal eigenfunctions are associated with the equilibrium dynamics, and their eigenvalues match those of the linearization of the nonlinear system at the equilibrium point. We exploit the structure of the principal eigenfunctions by decomposing them into linear and nonlinear components. The linear part corresponds to the left eigenvector of the system's linearization at the equilibrium, while the nonlinear part is obtained by solving a partial differential equation (PDE) using kernel methods. Our approach avoids common issues such as spectral pollution and spurious eigenvalues, which can arise in previous methods. We demonstrate the effectiveness of our algorithm through numerical examples.

math.DS↗

arXiv2024

Explaining the Explainers in Graph Neural Networks: a Comparative Study

Following a fast initial breakthrough in graph based learning, Graph Neural Networks (GNNs) have reached a widespread application in many science and engineering fields, prompting the need for methods to understand their decision process. GNN explainers have started to emerge in recent years, with a multitude of methods both novel or adapted from other domains. To sort out this plethora of alternative approaches, several studies have benchmarked the performance of different explainers in terms of various explainability metrics. However, these earlier works make no attempts at providing insights into why different GNN architectures are more or less explainable, or which explainer should be preferred in a given setting. In this survey, we fill these gaps by devising a systematic experimental study, which tests ten explainers on eight representative architectures trained on six carefully designed graph and node classification datasets. With our results we provide key insights on the choice and applicability of GNN explainers, we isolate key components that make them usable and successful and provide recommendations on how to avoid common interpretation pitfalls. We conclude by highlighting open questions and directions of possible future research.

cs.LG↗

arXiv2024

Separation Power of Equivariant Neural Networks

The separation power of a machine learning model refers to its ability to distinguish between different inputs and is often used as a proxy for its expressivity. Indeed, knowing the separation power of a family of models is a necessary condition to obtain fine-grained universality results. In this paper, we analyze the separation power of equivariant neural networks, such as convolutional and permutation-invariant networks. We first present a complete characterization of inputs indistinguishable by models derived by a given architecture. From this results, we derive how separability is influenced by hyperparameters and architectural choices-such as activation functions, depth, hidden layer width, and representation types. Notably, all non-polynomial activations, including ReLU and sigmoid, are equivalent in expressivity and reach maximum separation power. Depth improves separation power up to a threshold, after which further increases have no effect. Adding invariant features to hidden representations does not impact separation power. Finally, block decomposition of hidden representations affects separability, with minimal components forming a hierarchy in separation power that provides a straightforward method for comparing the separation power of models.

cs.LG↗

arXiv2024

Autonomous and Human-Driven Vehicles Interacting in a Roundabout: A Quantitative and Qualitative Evaluation

Optimizing traffic dynamics in an evolving transportation landscape is crucial, particularly in scenarios where autonomous vehicles (AVs) with varying levels of autonomy coexist with human-driven cars. While optimizing Reinforcement Learning (RL) policies for such scenarios is becoming more and more common, little has been said about realistic evaluations of such trained policies. This paper presents an evaluation of the effects of AVs penetration among human drivers in a roundabout scenario, considering both quantitative and qualitative aspects. In particular, we learn a policy to minimize traffic jams (i.e., minimize the time to cross the scenario) and to minimize pollution in a roundabout in Milan, Italy. Through empirical analysis, we demonstrate that the presence of AVs} can reduce time and pollution levels. Furthermore, we qualitatively evaluate the learned policy using a cutting-edge cockpit to assess its performance in near-real-world conditions. To gauge the practicality and acceptability of the policy, we conduct evaluations with human participants using the simulator, focusing on a range of metrics like traffic smoothness and safety perception. In general, our findings show that human-driven vehicles benefit from optimizing AVs dynamics. Also, participants in the study highlight that the scenario with 80% AVs is perceived as safer than the scenario with 20%. The same result is obtained for traffic smoothness perception.

cs.AI↗

arXiv2024

A Characterization Theorem for Equivariant Networks with Point-wise Activations

Equivariant neural networks have shown improved performance, expressiveness and sample complexity on symmetrical domains. But for some specific symmetries, representations, and choice of coordinates, the most common point-wise activations, such as ReLU, are not equivariant, hence they cannot be employed in the design of equivariant neural networks. The theorem we present in this paper describes all possible combinations of finite-dimensional representations, choice of coordinates and point-wise activations to obtain an exactly equivariant layer, generalizing and strengthening existing characterizations. Notable cases of practical relevance are discussed as corollaries. Indeed, we prove that rotation-equivariant networks can only be invariant, as it happens for any network which is equivariant with respect to connected compact groups. Then, we discuss implications of our findings when applied to important instances of exactly equivariant networks. First, we completely characterize permutation equivariant networks such as Invariant Graph Networks with point-wise nonlinearities and their geometric counterparts, highlighting a plethora of models whose expressive power and performance are still unknown. Second, we show that feature spaces of disentangled steerable convolutional neural networks are trivial representations.

cs.LG↗