arXiv subjects

Davide Bassetti

Publications and source records attributed to Davide Bassetti.

5 recordsLinked to original sources

arXiv2026

Distillation and Interpretability of Ensemble Forecasts of ENSO Phase using Entropic Learning

This paper introduces a distillation framework for an ensemble of entropy-optimal Sparse Probabilistic Approximation (eSPA) models, trained exclusively on satellite-era observational and reanalysis data to predict ENSO phase up to 24 months in advance. While eSPA ensembles yield state-of-the-art forecast skill, they are harder to interpret than individual eSPA models. We show how to compress the ensemble into a compact set of "distilled" models by aggregating the structure of only those ensemble members that make correct predictions. This process yields a single, diagnostically tractable model for each forecast lead time that preserves forecast performance while also enabling diagnostics that are impractical to implement on the full ensemble. An analysis of the regime persistence of the distilled model "superclusters", as well as cross-lead clustering consistency, shows that the discretised system accurately captures the spatiotemporal dynamics of ENSO. By considering the effective dimension of the feature importance vectors, the complexity of the input space required for correct ENSO phase prediction is shown to peak when forecasts must cross the boreal spring predictability barrier. Spatial importance maps derived from the feature importance vectors are introduced to identify where predictive information resides in each field and are shown to include known physical precursors at certain lead times. Case studies of key events are also presented, showing how fields reconstructed from distilled model centroids trace the evolution from extratropical and inter-basin precursors to the mature ENSO state. Overall, the distillation framework enables a rigorous investigation of long-range ENSO predictability that complements real-time data-driven operational forecasts.

physics.comp-ph↗

arXiv2025

An entropy-optimal path to humble AI

Progress of AI has led to very successful, but by no means humble models and tools, especially regarding (i) the huge and further exploding costs and resources they demand, and (ii) the over-confidence of these tools with the answers they provide. Here we introduce a novel mathematical framework for a non-equilibrium entropy-optimizing reformulation of Boltzmann machines based on the exact law of total probability and the exact convex polytope representations. We show that it results in the highly-performant, but much cheaper, gradient-descent-free learning framework with mathematically-justified existence and uniqueness criteria, and cheaply-computable confidence/reliability measures for both the model inputs and the outputs. Comparisons to state-of-the-art AI tools in terms of performance, cost and the model descriptor lengths on a broad set of synthetic and real-world problems with varying complexity reveal that the proposed method results in more performant and slim models, with the descriptor lengths being very close to the intrinsic complexity scaling bounds for the underlying problems. Applying this framework to historical climate data results in models with systematically higher prediction skills for the onsets of important La Ni\~na and El Ni\~no climate phenomena, requiring just few years of climate data for training - a small fraction of what is necessary for contemporary climate prediction tools.

cs.LG↗

arXiv2025

Fast, close, non-singular and property-preserving approximations of entropic measures

Entropic measures like Shannon entropy (SE), its quantum mechanical analogue von Neumann entropy, and Kullback-Leibler divergence (KL) are key components in many tools used in physics, information theory, machine learning (ML) and quantum computing. Besides of the significant amounts of SE and KL computations required in these fields, the singularity of their gradients near zero is one of the central mathematical reason inducing the high cost, frequently low robustness and slow convergence of computational tools that rely on these concepts. Here we propose the Fast Entropic Approximations (FEA) - non-singular rational approximations of SE and symmetrized KL, that preserve their main mathematical properties and achieve a mean absolute errors of around $10^-3$ ($10-20$ times better than comparable state-of-the-art computational approximations). We show that FEA allows up to around 2 times faster computation of SE and up to 37 times faster computation of symmetrized KL: it requires only $5$ to $7$ elementary computational operations, as compared to the tens of elementary operations behind SE and KL evaluations based on approximate logarithm schemes with table look-ups, bitshifts, or series approximations. On a set of common benchmarks for the feature selection problem in machine learning, we show that the combined effect of fewer elementary operations, low approximation error, preservation of main mathematical properties, and non-singular gradients allows much faster training of significantly-better models. We demonstrate that FEA enables ML feature extraction that is three orders of magnitude faster, and better in quality then the very popular LASSO feature extraction.

cs.LG↗

arXiv2025

Entropic learning enables skilful forecasts of ENSO phase at up to two years lead time

This paper extends previous work (Groom et al., \emph{Artif. Intell. Earth Syst.}, 2024) in applying the entropy-optimal Sparse Probabilistic Approximation (eSPA) algorithm to predict ENSO phase, defined by thresholding the Ni\~no3.4 index. Only satellite-era observational datasets are used for training and validation, while retrospective forecasts from 2012 to 2022 are used to assess out-of-sample skill at lead times up to 24 months. Rather than train a single eSPA model per lead, we introduce an ensemble approach in which multiple eSPA models are aggregated via a novel meta-learning strategy. The features used include the leading principal components from a delay-embedded EOF analysis of global sea surface temperature, vertical temperature gradient (a thermocline proxy), and tropical Pacific wind stresses. Crucially, the data is processed to prevent any form of information leakage from the future, ensuring realistic real-time forecasting conditions. Despite the limited number of training instances, eSPA avoids overfitting and produces probabilistic forecasts with skill comparable to the International Research Institute for Climate and Society (IRI) ENSO prediction plume. Beyond the IRI's lead times, eSPA maintains skill out to 22 months for the ranked probability skill score and 24 months for accuracy and area under the ROC curve, all at a fraction of the computational cost of a fully-coupled dynamical model. Furthermore, eSPA successfully forecasts the 2015/16 and 2018/19 El Ni\~no events at 24 months lead, the 2016/17, 2017/18 and 2020/21 La Ni\~na events at 24 months lead and the 2021/22 and 2022/23 La Ni\~na events at 12 and 8 months lead.

physics.comp-ph↗

arXiv2023

Gauge-optimal approximate learning for small data classification problems

Small data learning problems are characterized by a significant discrepancy between the limited amount of response variable observations and the large feature space dimension. In this setting, the common learning tools struggle to identify the features important for the classification task from those that bear no relevant information, and cannot derive an appropriate learning rule which allows to discriminate between different classes. As a potential solution to this problem, here we exploit the idea of reducing and rotating the feature space in a lower-dimensional gauge and propose the Gauge-Optimal Approximate Learning (GOAL) algorithm, which provides an analytically tractable joint solution to the dimension reduction, feature segmentation and classification problems for small data learning problems. We prove that the optimal solution of the GOAL algorithm consists in piecewise-linear functions in the Euclidean space, and that it can be approximated through a monotonically convergent algorithm which presents -- under the assumption of a discrete segmentation of the feature space -- a closed-form solution for each optimization substep and an overall linear iteration cost scaling. The GOAL algorithm has been compared to other state-of-the-art machine learning (ML) tools on both synthetic data and challenging real-world applications from climate science and bioinformatics (i.e., prediction of the El Nino Southern Oscillation and inference of epigenetically-induced gene-activity networks from limited experimental data). The experimental results show that the proposed algorithm outperforms the reported best competitors for these problems both in learning performance and computational cost.

cs.LG↗