arXiv subjects

Enrique Zuazua

Publications and source records attributed to Enrique Zuazua.

At least 37 records · Page 2Linked to original sources

arXiv2026

Sherpa.ai Privacy-Preserving Multi-Party Entity Alignment without Intersection Disclosure for Noisy Identifiers

Federated Learning (FL) enables collaborative model training among multiple parties without centralizing raw data. There are two main paradigms in FL: Horizontal FL (HFL), where all participants share the same feature space but hold different samples, and Vertical FL (VFL), where parties possess complementary features for the same set of samples. A prerequisite for VFL training is privacy-preserving entity alignment (PPEA), which establishes a common index of samples across parties (alignment) without revealing which samples are shared between them. Conventional private set intersection (PSI) achieves alignment but leaks intersection membership, exposing sensitive relationships between datasets. The standard private set union (PSU) mitigates this risk by aligning on the union of identifiers rather than the intersection. However, existing approaches are often limited to two parties or lack support for typo-tolerant matching. In this paper, we introduce the Sherpa.ai multi-party PSU protocol for VFL, a PPEA method that hides intersection membership and enables both exact and noisy matching. The protocol generalizes two-party approaches to multiple parties with low communication overhead and offers two variants: an order-preserving version for exact alignment and an unordered version tolerant to typographical and formatting discrepancies. We prove correctness and privacy, analyze communication and computational (exponentiation) complexity, and formalize a universal index mapping from local records to a shared index space. This multi-party PSU offers a scalable, mathematically grounded protocol for PPEA in real-world VFL deployments, such as multi-institutional healthcare disease detection, collaborative risk modeling between banks and insurers, and cross-domain fraud detection between telecommunications and financial institutions, while preserving intersection privacy.

cs.CR↗

arXiv2026

Approximate and Weighted Data Reconstruction Attack in Federated Learning

Federated Learning (FL) is a distributed learning paradigm that enables multiple clients to collaborate on building a machine learning model without sharing their private data. Although FL is considered privacy-preserved by design, recent data reconstruction attacks demonstrate that an attacker can recover clients' training data based on the parameters shared in FL. However, most existing methods fail to attack the most widely used horizontal Federated Averaging (FedAvg) scenario, where clients share model parameters after multiple local training steps. To tackle this issue, we propose an interpolation-based approximation method, which makes attacking FedAvg scenarios feasible by generating the intermediate model updates of the clients' local training processes. Then, we design a layer-wise weighted loss function to improve the data quality of reconstruction. We assign different weights to model updates in different layers concerning the neural network structure, with the weights tuned by Bayesian optimization. Finally, experimental results validate the superiority of our proposed approximate and weighted attack (AWA) method over the other state-of-the-art methods, as demonstrated by the substantial improvement in different evaluation metrics for image data reconstructions.

cs.LG↗

arXiv2026

Towards the Next Frontier of LLMs, Training on Private Data: A Cross-Domain Benchmark for Federated Fine-Tuning

The recent success of large language models (LLMs) has been largely driven by vast public datasets. However, the next frontier for LLM development lies beyond public data. Much of the world's most valuable information is private, especially in highly regulated sectors such as healthcare and finance, where data include patient histories or customer communications. Unlocking this data could represent a major leap forward, enabling LLMs with deeper domain expertise and stronger real-world utility. Yet, these data cannot be shared because they are distributed across institutions and constrained by privacy, regulatory, and organizational barriers. Moreover, institutional datasets are typically non-independent and identically distributed (non-IID), differing across sites in population characteristics, data modalities, documentation patterns, and task-specific label distributions. In this paper, we demonstrate a practical approach to unlocking private and distributed institutional data for LLM adaptation through federated collaboration across data silos. Built on the Sherpa.ai Federated Learning platform, our framework enables nodes to jointly fine-tune a shared LLM without exchanging private data. We evaluate this approach through a cross-domain benchmark in healthcare and finance, using four closed-ended question answering and classification datasets: MedQA, MedMCQA, FPB, and FiQA-SA. We compare three parameter-efficient fine-tuning (PEFT) strategies-LoRA, QLoRA, and IA3-across pretrained backbones under non-IID settings reflecting institutional data heterogeneity. Our results show that federated fine-tuning performs close to centralized training and outperforms isolated single-institution learning. From a Green AI perspective, QLoRA and IA3 improve efficiency with limited accuracy degradation, supporting federated PEFT as a viable approach for adapting LLMs where data cannot be shared.

cs.LG↗

arXiv2026

Exact Sequence Interpolation with Transformers

We prove that transformers can exactly interpolate datasets of finite input sequences in $\mathbb{R}^d$, $d\geq 2$, with corresponding output sequences of smaller or equal length. Specifically, given $N$ sequences of arbitrary but finite lengths in $\mathbb{R}^d$ and output sequences of lengths $m^1, \dots, m^N \in \mathbb{N}$, we construct a transformer with $\mathcal{O}(\sum_{j=1}^N m^j)$ blocks and $\smash{\mathcal{O}(d \sum_{j=1}^N m^j)}$ parameters that exactly interpolates the dataset. Our construction provides complexity estimates that are independent of the input sequence length, by alternating feed-forward and self-attention layers and by capitalizing on the clustering effect inherent to the latter. Our novel constructive method also uses low-rank parameter matrices in the self-attention mechanism, a common feature of practical transformer implementations. These results are first established in the hardmax self-attention setting, where the geometric structure permits an explicit and quantitative analysis, and are then extended to the softmax setting. Finally, we demonstrate the applicability of our exact interpolation construction to learning problems, in particular by providing convergence guarantees to a global minimizer under regularized training strategies. Our analysis contributes to the theoretical understanding of transformer models, offering an explanation for their excellent performance in exact sequence-to-sequence interpolation tasks.

cs.LG↗

arXiv2026

Clustering in pure-attention hardmax transformers and its role in sentiment analysis

Transformers are extremely successful machine learning models whose mathematical properties remain poorly understood. Here, we rigorously characterize the behavior of transformers with hardmax self-attention and normalization sublayers as the number of layers tends to infinity. By viewing such transformers as discrete-time dynamical systems describing the evolution of points in a Euclidean space, and thanks to a geometric interpretation of the self-attention mechanism based on hyperplane separation, we show that the transformer inputs asymptotically converge to a clustered equilibrium determined by special points called \textit{leaders}. We then leverage this theoretical understanding to solve sentiment analysis problems from language processing using a fully interpretable transformer model, which effectively captures `context' by clustering meaningless words around leader words carrying the most meaning. Finally, we outline remaining challenges to bridge the gap between the mathematical analysis of transformers and their real-life implementation.

cs.CL↗

arXiv2026

HYCO: A Formalism for Hybrid-Cooperative PDE Modelling

We present Hybrid-Cooperative Learning (HYCO), a hybrid modeling framework that integrates physics-based and data-driven models through mutual regularization. Unlike traditional approaches that impose physical constraints directly on synthetic models, HYCO treats both components as co-trained agents nudged toward agreement. This cooperative scheme is naturally parallelizable and demonstrates robustness to sparse and noisy data. Numerical experiments on static and time-dependent benchmark problems show that HYCO can recover accurate solutions and model parameters under ill-posed conditions. The framework admits a game-theoretic interpretation as a Nash equilibrium problem, enabling alternating optimization. This paper is based on the extended preprint: arXiv:2509.14123 .

math.OC↗

arXiv2026

Geometric Asymptotics of Score Mixing and Guidance in Diffusion Models

Diffusion models are routinely guided in practice by combining multiple score fields, yet the mathematical structure of score mixing is still poorly understood. We study the small-time generation dynamics driven by mixed scores $$ s=λ\,\nabla\log u_1+(1-λ)\,\nabla\log u_2,\qquad λ\ge 0, $$ in the heat-flow framework, where $u_1,u_2$ are heat evolutions of two compactly supported probability measures. This single formulation covers both the mixture-of-experts regime $(0\leq λ\leq 1)$ and the classifier-free guidance regime $(λ>1)$. Exploiting a Laplace-Varadhan principle under a similarity-time rescaling, we show that the small-time generation dynamics is governed by the explicit geometric potential $$ Φ_λ=λd_1^2+(1-λ)d_2^2, $$ which depends only on the supports of the initial measures and on the mixing parameter. This gives a rigorous reduction from a singular, non-autonomous score-driven dynamics to autonomous Clarke-type subgradient inclusions. In the empirical setting of finite Dirac mixtures, the limiting potential is piecewise quadratic with a Voronoi-type structure; this rigidity yields convergence of all autonomous limiting trajectories to critical points and a conditional convergence criterion for the original generation flow toward local minimizers of the potential, with rate $\mathcal O(\sqrt t)$ in the smooth stable case.

math.OC↗

arXiv2026

Hamiltonian Interface Dynamics for Reduced-Order Optimization of Incompressible Mixing

We develop a reduced-order framework for optimizing mixing in two-dimensional incompressible flows. Instead of optimizing the full transport PDE, the method maximizes the length of advected material interfaces, leading to a finite-dimensional Hamiltonian control problem based on parametrized stream functions. We derive the continuous adjoint equations and reduced gradients, and discretize the forward and adjoint dynamics with the implicit midpoint rule. The resulting discrete adjoint is algebraically consistent with the derivative of the fully discrete objective, up to the tolerance of the nonlinear midpoint solves. The approach applies to bounded two-dimensional domains with smooth finite-dimensional stream-function parametrizations. Numerical experiments on cellular-flow and Doswell frontogenesis benchmarks show that the optimized time-dependent Hamiltonians generate near-exponential interface stretching and substantially faster decay of the $\dot{H}^{-1}$ mix-norm, in contrast with the polynomial behavior observed for stationary flows. When evaluated on a common reference transport solver, the interface-based controls produce faster $\dot{H}^{-1}$ decay than a Eulerian Sobolev-norm optimizer under a matched setup, while substantially reducing computational cost. We also identify a limitation of the reduced model: increasing the control basis may further improve the interface-length objective without yielding proportional gains in $\dot{H}^{-1}$ mixing, confirming that interface length is an effective but not fully faithful proxy for mixing in geometrically complex regimes.

math.NA↗

arXiv2026

Computational performance of the MMOC in the inverse design of the Doswell frontogenesis equation

Inverse design of transport equations can be addressed by using a gradient-adjoint methodology. In this methodology numerical schemes used for the adjoint resolution determine the direction of descent in its iterative algorithm, and consequently the CPU time consumed by the inverse design. As the CPU time constitutes a known bottleneck, it is important to employ light and quick schemes to the adjoint problem. In this regard, we proposed to use the Modified Method of Characteristics (MMOC). Despite not preserving identity conservation, the MMOC is computationally competitive. In this work we investigated the advantage of using the MMOC in comparison with the Lax-Friedrichs and Lax-Wendro? schemes for the inverse design problem. By testing the Doswell frontogenesis equation, we observed that the MMOC can provide more efficient and accurate computation under some simulation conditions.

math.NA↗

arXiv2026

Fair feature attribution for multi-output prediction: a Shapley-based perspective

In this article, we provide an axiomatic characterization of feature attribution for multi-output predictors within the Shapley framework. While SHAP explanations are routinely computed independently for each output coordinate, the theoretical necessity of this practice has remained unclear. By extending the classical Shapley axioms to vector-valued cooperative games, we establish a rigidity theorem showing that any attribution rule satisfying efficiency, symmetry, dummy player, and additivity must necessarily decompose component-wise across outputs. Consequently, any joint-output attribution rule must relax at least one of the classical Shapley axioms. This result identifies a previously unformalized structural constraint in Shapley-based interpretability, clarifying the precise scope of fairness-consistent explanations in multi-output learning. Numerical experiments on a biomedical benchmark illustrate that multi-output models can yield computational savings in training and deployment, while producing SHAP explanations that remain fully consistent with the component-wise structure imposed by the Shapley axioms.

cs.LG↗

arXiv2026

Sensor placement via large deviations in the Eikonal equation

In this work, we address the problem of optimally placing a finite number of sensors within a given region so as to minimize the mean or maximal distance to the points of the domain. To tackle this natural geometric performance criterion, formulated in terms of distance functions, we combine tools from geometric analysis with a classical result of Varadhan, which provides an efficient approximation of the distance function via the solution of a simple elliptic PDE. The effectiveness of the proposed approach is demonstrated through illustrative numerical simulations.

math.OC↗

arXiv2026

Moments, Time-Inversion and Source Identification for the Heat Equation

We address the initial source identification problem for the heat equation, a notably ill-posed inverse problem characterized by exponential instability. Departing from classical Tikhonov regularization, we propose a novel approach based on moment analysis of the heat flow, transforming the problem into a more stable inverse moment formulation. By evolving the measured terminal time moments backward through their governing ODE system, we recover the moments of the initial distribution. We then reconstruct the source by solving a convex optimization problem that minimizes the total variation of a measure subject to these moment constraints. This formulation naturally promotes sparsity, yielding atomic solutions that are sums of Dirac measures. Compared to existing methods, our moment-based approach reduces exponential error growth to polynomial growth with respect to the terminal time. We provide explicit error estimates on the recovered initial distributions in terms of moment order, terminal time, and measurement errors. In addition, we develop efficient numerical discretization schemes and demonstrate significant stability improvements of our approach through comprehensive numerical experiments.

math.NA↗

arXiv2026

Universal Approximation of Dynamical Systems by Semi-Autonomous Neural ODEs and Applications

In this paper, we introduce semi-autonomous neural ordinary differential equations (SA-NODEs), a variation of the vanilla NODEs, employing fewer parameters. We investigate the universal approximation properties of SA-NODEs for dynamical systems from both a theoretical and a numerical perspective. Within the assumption of a finite-time horizon, under general hypotheses we establish an asymptotic approximation result, demonstrating that the error vanishes as the number of parameters goes to infinity. Under additional regularity assumptions, we further specify this convergence rate in relation to the number of parameters, utilizing quantitative approximation results in the Barron space. Based on the previous result, we prove an approximation rate for transport equations by their neural counterparts. Our numerical experiments validate the effectiveness of SA-NODEs in capturing the dynamics of various ODE systems and transport equations. Additionally, we compare SA-NODEs with vanilla NODEs, highlighting the superior performance and reduced complexity of our approach.

math.NA↗

arXiv2026

A Structure-Preserving Numerical Scheme for Optimal Control and Design of Mixing in Incompressible Flows

We develop a structure-preserving computational framework for optimal mixing control in incompressible flows. Our approach exactly conserves the continuous system's key invariants (mass and $L^2$-energy), while also maintaining discrete state-adjoint duality at every time step. These properties are achieved by integrating a centered finite-volume discretization in space with a time-symmetric Crank-Nicolson integrator for both the forward advection and its adjoint, all inside a gradient-based optimization loop. The result is a numerical solver that is faithful to the continuous optimality conditions and efficiently computes mixing-enhancing controls. In our numerical tests, the optimized time-dependent stirring produces a nearly exponential decay of a chosen mix-norm, achieving orders-of-magnitude faster mixing than any single steady flow. To our knowledge, this work provides the first evidence that enforcing physical structure at the discrete level can lead to both exact conservation and highly effective mixing outcomes in optimal flow design.

math.NA↗

arXiv2025

Training Together, Diagnosing Better: Federated Learning for Collagen VI-Related Dystrophies

The application of Machine Learning (ML) to the diagnosis of rare diseases, such as collagen VI-related dystrophies (COL6-RD), is fundamentally limited by the scarcity and fragmentation of available data. Attempts to expand sampling across hospitals, institutions, or countries with differing regulations face severe privacy, regulatory, and logistical obstacles that are often difficult to overcome. The Federated Learning (FL) provides a promising solution by enabling collaborative model training across decentralized datasets while keeping patient data local and private. Here, we report a novel global FL initiative using the Sherpa.ai FL platform, which leverages FL across distributed datasets in two international organizations for the diagnosis of COL6-RD, using collagen VI immunofluorescence microscopy images from patient-derived fibroblast cultures. Our solution resulted in an ML model capable of classifying collagen VI patient images into the three primary pathogenic mechanism groups associated with COL6-RD: exon skipping, glycine substitution, and pseudoexon insertion. This new approach achieved an F1-score of 0.82, outperforming single-organization models (0.57-0.75). These results demonstrate that FL substantially improves diagnostic utility and generalizability compared to isolated institutional models. Beyond enabling more accurate diagnosis, we anticipate that this approach will support the interpretation of variants of uncertain significance and guide the prioritization of sequencing strategies to identify novel pathogenic variants.

cs.LG↗

arXiv2025

Machine Learning and Control: Foundations, Advances, and Perspectives

Control theory of dynamical systems offers a powerful framework for tackling challenges in deep neural networks and other machine learning architectures. We show that concepts such as simultaneous and ensemble controllability offer new insights into the classification and representation properties of deep neural networks, while the control and optimization of static systems can be employed to better understand the performance of shallow networks. Inspired by the classical concept of turnpike, we also explore the relationship between dynamic and static neural networks, where depth is traded for width, and the role of transformers as mechanisms for accelerating classical neural network tasks. We also exploit the expressive power of neural networks (exemplified, for instance, by the Universal Approximation Theorem) to develop a novel hybrid modeling methodology, the Hybrid-Cooperative Learning (HYCO), combining mechanics and data-driven methods in a game-theoretic setting. Finally, we describe how classical properties of diffusion processes, long established in the context of partial differential equations, contribute to explaining the success of modern generative artificial intelligence (AI). We present an overview of our recent results in these areas, illustrating how control, machine learning, numerical analysis, and partial differential equations come together to motivate a fertile ground for future research.

math.OC↗

arXiv2025

Mean-Square Stability of Continuous-Time Stochastic Model Predictive Control

We propose a stochastic model predictive control (SMPC) framework for a broad class of unconstrained controlled stochastic differential equations (SDEs) and establish its mean-square exponential stability in the infinite-horizon limit. At each prediction step of the MPC iteration, the nonlinear controlled SDE is approximated by its linearization at the origin, with the sampled state of the nonlinear system as initial condition, yielding a finite-horizon stochastic linear-quadratic (SLQ) optimal control problem. The resulting optimal control is then applied to the original nonlinear stochastic dynamics until the next sampling instant. This construction leads to a delayed SMPC scheme whose closed-loop behavior is governed by a coupled time-delay SDE system, a setting that has not been analyzed before. We prove global mean-square exponential stability for linear and mildly nonlinear SDEs by exploiting the exponential convergence of the Riccati equation to the algebraic Riccati equation (ARE). For strongly nonlinear SDEs, we establish local mean-square exponential stability by combining exponential Riccati convergence with stopping-time techniques and Grönwall-type estimates. It is observed that, to ensure the desired local stability properties, the nonlinearities of the SDE are allowed to have polynomial growth but not exponential growth, distinguishing SMPC from its deterministic counterpart. These results provide the first rigorous mean-square stability guarantees for SMPC of SDE systems with delayed state information, thereby advancing the theoretical foundations of stochastic predictive control.

math.OC↗

arXiv2025

Federated Learning for Pediatric Pneumonia Detection: Enabling Collaborative Diagnosis Without Sharing Patient Data

Early and accurate pneumonia detection from chest X-rays (CXRs) is clinically critical to expedite treatment and isolation, reduce complications, and curb unnecessary antibiotic use. Although artificial intelligence (AI) substantially improves CXR-based detection, development is hindered by globally distributed data, high inter-hospital variability, and strict privacy regulations (e.g., HIPAA, GDPR) that make centralization impractical. These constraints are compounded by heterogeneous imaging protocols, uneven data availability, and the costs of transferring large medical images across geographically dispersed sites. In this paper, we evaluate Federated Learning (FL) using the Sherpa.ai FL platform, enabling multiple hospitals (nodes) to collaboratively train a CXR classifier for pneumonia while keeping data in place and private. Using the Pediatric Pneumonia Chest X-ray dataset, we simulate cross-hospital collaboration with non-independent and non-identically distributed (non-IID) data, reproducing real-world variability across institutions and jurisdictions. Our experiments demonstrate that collaborative and privacy-preserving training across multiple hospitals via FL led to a dramatic performance improvement achieving 0.900 Accuracy and 0.966 ROC-AUC, corresponding to 47.5% and 50.0% gains over single-hospital models (0.610; 0.644), without transferring any patient CXR. These results indicate that FL delivers high-performing, generalizable, secure and private pneumonia detection across healthcare networks, with data kept local. This is especially relevant for rare diseases, where FL enables secure multi-institutional collaboration without data movement, representing a breakthrough for accelerating diagnosis and treatment development in low-data domains.

cs.LG↗