arXiv subjects

Guido Sanguinetti

Publications and source records attributed to Guido Sanguinetti.

At least 19 recordsLinked to original sources

arXiv2026

BayesPrompt: human readable prompts that make sense

Reconstructing prompts that can elicit a desired answer or behaviour in an LLM is an open and important research topic. Optimisation methods which aim at minimising the perplexity of a given answer, however, consistently yield so-called pseudoprompts, unintelligible strings of tokens which can lack human interpretability. We argue that this is a consequence of the ill-posedness of the prompt optimisation task. By reframing the task as a Bayesian posterior inference over prompts, we propose an efficient algorithm to sample prompts which are both efficient (in terms of perplexity) and human readable. We compare our approach with state of the art alternatives showing on a real data set a marked improvement over a range of metrics.

cs.CL↗

arXiv2026

MERGE-RNA: a physics-based model to predict RNA secondary structure ensembles with chemical probing

RNA function is tied to secondary structure, operating through dynamic and heterogeneous structural ensembles. While current analysis tools typically output single static structures or averaged contact maps, chemical probing methods like DMS capture nucleotide-resolution signals representing the full structural ensemble, which remain difficult to interpret structurally. To address this, we present MERGE-RNA, a framework that describes and outputs RNA as a structural ensemble. By modeling the physics of the experimental pipeline, MERGE-RNA learns a small set of transferable and interpretable parameters, enabling the integration of measurements across different molecules, probe concentrations, and replicates in a single optimization to improve robustness. Our model employs a maximum-entropy principle to predict thermodynamic populations, with the minimal adjustments necessary to align the ensemble with experimental data. We validate MERGE-RNA on diverse RNAs, showing that it achieves structural accuracy surpassing standard pseudo-free-energy methods and yields ensembles better recapitulating measured DMS reactivity. Applied to the V. vulnificus adenine riboswitch, MERGE-RNA recovers the NMR-resolved conformations and their ligand-induced rearrangement, with population shifts matching the NMR-derived K_d. In a designed RNA construct for which we report new DMS data, MERGE-RNA deconvolves mixed states to reveal transient intermediate populations involved in strand displacement, dynamics invisible to methods based on enumerating a small number of structures.

q-bio.BM↗

arXiv2026

Machine Learning for RNA Secondary Structure Prediction: a review of current methods and challenges

Predicting the secondary structure of RNA is a core challenge in computational biology, essential for understanding molecular function and designing novel therapeutics. The field has evolved from foundational but accuracy-limited thermodynamic approaches to a new data-driven paradigm dominated by machine learning and deep learning. These models learn folding patterns directly from data, leading to significant performance gains. This review surveys the modern landscape of these methods, covering single-sequence, evolutionary-based, and hybrid models that blend machine learning with biophysics. A central theme is the field's "generalization crisis," where powerful models were found to fail on new RNA families, prompting a community-wide shift to stricter, homology-aware benchmarking. In response to the underlying challenge of data scarcity, RNA foundation models have emerged, learning from massive, unlabeled sequence corpora to improve generalization. Finally, we look ahead to the next set of major hurdles-including the accurate prediction of complex motifs like pseudoknots, scaling to kilobase-length transcripts, incorporating the chemical diversity of modified nucleotides, and shifting the prediction target from static structures to the dynamic ensembles that better capture biological function. We also highlight the need for a standardized, prospective benchmarking system to ensure unbiased validation and accelerate progress.

q-bio.BM↗

arXiv2026

CoVAE: correlated multimodal generative modeling

Multimodal Variational Autoencoders have emerged as a popular tool to extract effective representations from rich multimodal data. However, such models rely on fusion strategies in latent space that destroy the joint statistical structure of the multimodal data, with profound implications for generation and uncertainty quantification. In this work, we introduce Correlated Variational Autoencoders (CoVAE), a new generative architecture that captures the correlations between modalities. We test CoVAE on a number of real and synthetic data sets demonstrating both accurate cross-modal reconstruction and effective quantification of the associated uncertainties.

cs.LG↗

arXiv2025

Balanced Multimodal Learning via Mutual Information

Multimodal learning has increasingly become a focal point in research, primarily due to its ability to integrate complementary information from diverse modalities. Nevertheless, modality imbalance, stemming from factors such as insufficient data acquisition and disparities in data quality, has often been inadequately addressed. This issue is particularly prominent in biological data analysis, where datasets are frequently limited, costly to acquire, and inherently heterogeneous in quality. Conventional multimodal methodologies typically fall short in concurrently harnessing intermodal synergies and effectively resolving modality conflicts. In this study, we propose a novel unified framework explicitly designed to address modality imbalance by utilizing mutual information to quantify interactions between modalities. Our approach adopts a balanced multimodal learning strategy comprising two key stages: cross-modal knowledge distillation (KD) and a multitask-like training paradigm. During the cross-modal KD pretraining phase, stronger modalities are leveraged to enhance the predictive capabilities of weaker modalities. Subsequently, our primary training phase employs a multitask-like learning mechanism, dynamically calibrating gradient contributions based on modality-specific performance metrics and intermodal mutual information. This approach effectively alleviates modality imbalance, thereby significantly improving overall multimodal model performance.

cs.LG↗

arXiv2025

Information-Theoretic Criteria for Knowledge Distillation in Multimodal Learning

The rapid increase in multimodal data availability has sparked significant interest in cross-modal knowledge distillation (KD) techniques, where richer "teacher" modalities transfer information to weaker "student" modalities during model training to improve performance. However, despite successes across various applications, cross-modal KD does not always result in improved outcomes, primarily due to a limited theoretical understanding that could inform practice. To address this gap, we introduce the Cross-modal Complementarity Hypothesis (CCH): we propose that cross-modal KD is effective when the mutual information between teacher and student representations exceeds the mutual information between the student representation and the labels. We theoretically validate the CCH in a joint Gaussian model and further confirm it empirically across diverse multimodal datasets, including image, text, video, audio, and cancer-related omics data. Our study establishes a novel theoretical framework for understanding cross-modal KD and offers practical guidelines based on the CCH criterion to select optimal teacher modalities for improving the performance of weaker modalities.

cs.LG↗

arXiv2025

bioSBM: a random graph model to integrate epigenomic data in chromatin structure prediction

The spatial organization of chromatin within the nucleus plays a crucial role in gene expression and genome function. However, the quantitative relationship between this organization and nuclear biochemical processes remains under debate. In this study, we present a graph-based generative model, bioSBM, designed to capture long-range chromatin interaction patterns from Hi-C data and, importantly, simultaneously link these patterns to biochemical features. Applying bioSBM to Hi-C maps of the GM12878 lymphoblastoid cell line, we identified a latent structure of chromatin interactions, revealing 7 distinct communities that strongly align with known biological annotations. Additionally, we infer a linear transformation that maps biochemical observables, such as histone marks, to the parameters of the generative graph model, enabling accurate genome-wide predictions of chromatin contact maps on out-of-sample data, both within the same cell line, and on the completely unseen HCT116 cell line under RAD21 depletion. These findings highlight bioSBM's potential as a powerful tool for elucidating the relationship between biochemistry and chromatin architecture and predicting long-range genome organization from independent biochemical data.

q-bio.QM↗

arXiv2025

Navigation through Non-Compact Symmetric Spaces: a mathematical perspective on Cartan Neural Networks

Recent work has identified non-compact symmetric spaces U/H as a promising class of homogeneous manifolds to develop a geometrically consistent theory of neural networks. An initial implementation of these concepts has been presented in a twin paper under the moniker of Cartan Neural Networks, showing both the feasibility and the performance of these geometric concepts in a machine learning context. The current paper expands on the mathematical structures underpinning Cartan Neural Networks, detailing the geometric properties of the layers and how the maps between layers interact with such structures to make Cartan Neural Networks covariant and geometrically interpretable. Together, these twin papers constitute a first step towards a fully geometrically interpretable theory of neural networks exploiting group-theoretic structures

cs.LG↗

arXiv2025

Cartan Networks: Group theoretical Hyperbolic Deep Learning

Hyperbolic deep learning leverages the metric properties of hyperbolic spaces to develop efficient and informative embeddings of hierarchical data. Here, we focus on the solvable group structure of hyperbolic spaces, which follows naturally from their construction as symmetric spaces. This dual nature of Lie group and Riemannian manifold allows us to propose a new class of hyperbolic deep learning algorithms where group homomorphisms are interleaved with metric-preserving diffeomorphisms. The resulting algorithms, which we call Cartan networks, show promising results on various benchmark data sets and open the way to a novel class of hyperbolic deep learning architectures.

cs.LG↗

arXiv2025

BayVel: A Bayesian Framework for RNA Velocity Estimation in Single-Cell Transcriptomics

RNA velocity is a model of gene expression dynamics designed to analyze single-cell RNA sequencing (scRNA-seq) data, and it has recently gained significant attention. However, despite its popularity, the model has raised several concerns, primarily related to three issues: its heavy dependence on data preprocessing, the need for post-processing of the results, and the limitations of the underlying statistical methodology. Current approaches, such as scVelo, suffer from notable statistical shortcomings. These include identifiability problems, reliance on heuristic preprocessing steps, and the absence of uncertainty quantification. To address these limitations, we propose BayVel, a Bayesian hierarchical model that directly models raw count data. BayVel resolves identifiability issues and provides posterior distributions for all parameters, including the RNA velocities themselves, without the need for any post processing. We evaluate BayVel's performance using simulated datasets. While scVelo fails to accurately reconstruct parameters, even when data are simulated directly from the model assumptions, BayVel demonstrates strong accuracy and robustness. This highlights BayVel as a statistically rigorous and reliable framework for studying transcriptional dynamics in the context of RNA velocity modeling. When applied to a real dataset of pancreatic epithelial cells previously analyzed with scVelo, BayVel does not replicate their findings, which appears to be strongly influenced by the postprocessing, supporting concerns raised in other studies about the reliability of scVelo.

stat.AP↗

arXiv2024

On the Robustness of Bayesian Neural Networks to Adversarial Attacks

Vulnerability to adversarial attacks is one of the principal hurdles to the adoption of deep learning in safety-critical applications. Despite significant efforts, both practical and theoretical, training deep learning models robust to adversarial attacks is still an open problem. In this paper, we analyse the geometry of adversarial attacks in the large-data, overparameterized limit for Bayesian Neural Networks (BNNs). We show that, in the limit, vulnerability to gradient-based attacks arises as a result of degeneracy in the data distribution, i.e., when the data lies on a lower-dimensional submanifold of the ambient space. As a direct consequence, we demonstrate that in this limit BNN posteriors are robust to gradient-based adversarial attacks. Crucially, we prove that the expected gradient of the loss with respect to the BNN posterior distribution is vanishing, even when each neural network sampled from the posterior is vulnerable to gradient-based attacks. Experimental results on the MNIST, Fashion MNIST, and half moons datasets, representing the finite data regime, with BNNs trained with Hamiltonian Monte Carlo and Variational Inference, support this line of arguments, showing that BNNs can display both high accuracy on clean data and robustness to both gradient-based and gradient-free based adversarial attacks.

cs.LG↗

arXiv2023

Attacks on Online Learners: a Teacher-Student Analysis

Machine learning models are famously vulnerable to adversarial attacks: small ad-hoc perturbations of the data that can catastrophically alter the model predictions. While a large literature has studied the case of test-time attacks on pre-trained models, the important case of attacks in an online learning setting has received little attention so far. In this work, we use a control-theoretical perspective to study the scenario where an attacker may perturb data labels to manipulate the learning dynamics of an online learner. We perform a theoretical analysis of the problem in a teacher-student setup, considering different attack strategies, and obtaining analytical results for the steady state of simple linear learners. These results enable us to prove that a discontinuous transition in the learner's accuracy occurs when the attack strength exceeds a critical threshold. We then study empirically attacks on learners with complex architectures using real data, confirming the insights of our theoretical analysis. Our findings show that greedy attacks can be extremely efficient, especially when data stream in small batches.

stat.ML↗

arXiv2023

Quantifying lottery tickets under label noise: accuracy, calibration, and complexity

Pruning deep neural networks is a widely used strategy to alleviate the computational burden in machine learning. Overwhelming empirical evidence suggests that pruned models retain very high accuracy even with a tiny fraction of parameters. However, relatively little work has gone into characterising the small pruned networks obtained, beyond a measure of their accuracy. In this paper, we use the sparse double descent approach to identify univocally and characterise pruned models associated with classification tasks. We observe empirically that, for a given task, iterative magnitude pruning (IMP) tends to converge to networks of comparable sizes even when starting from full networks with sizes ranging over orders of magnitude. We analyse the best pruned models in a controlled experimental setup and show that their number of parameters reflects task difficulty and that they are much better than full networks at capturing the true conditional probability distribution of the labels. On real data, we similarly observe that pruned models are less prone to overconfident predictions. Our results suggest that pruned models obtained via IMP not only have advantageous computational properties but also provide a better representation of uncertainty in learning.

cs.LG↗

arXiv2023

Bottom-up data integration in polymer models of chromatin organisation

Cellular functions crucially depend on the precise execution of complex biochemical reactions taking place on the chromatin fiber in the tightly packed environment of the cell nucleus. Despite the availability of large data sets probing this process from multiple angles, we still lack a bottom-up framework which can incorporate the sequence-specific nature of biochemistry in a unified model of 3D chromatin dynamics. Here we propose SEMPER (Sequence Enhanced Magnetic PolymER), a novel stochastic polymer model which naturally incorporates observational data about sequence-driven biochemical processes, such as binding of transcription factor proteins, in a 3D model of chromatin structure. By introducing a new algorithm for approximate Bayesian inference, we discuss how to estimate in a robust manner the relative importance of biochemical vs. polymer signals in the determination of the chromatin epigenetic states which is leading to a significant revision of the interpretation of previous models. Furthermore we show that, without additional input from the genome 3D structure, our model can predict with reasonable accuracy some notable and non trivial conformational features of chromatin folding within the nucleus. Our work highlights the importance of introducing physically realistic statistical models for predicting chromatin states from epigenetic data, and opens the way to a new class of more systematic approaches to interpret epigenomic data.

q-bio.QM↗

arXiv2022

Bayesian learning of effective chemical master equations in crowded intracellular conditions

Biochemical reactions inside living cells often occur in the presence of crowders -- molecules that do not participate in the reactions but influence the reaction rates through excluded volume effects. However the standard approach to modelling stochastic intracellular reaction kinetics is based on the chemical master equation (CME) whose propensities are derived assuming no crowding effects. Here, we propose a machine learning strategy based on Bayesian Optimisation utilising synthetic data obtained from spatial cellular automata (CA) simulations (that explicitly model volume-exclusion effects) to learn effective propensity functions for CMEs. The predictions from a small CA training data set can then be extended to the whole range of parameter space describing physiologically relevant levels of crowding by means of Gaussian Process regression. We demonstrate the method on an enzyme-catalyzed reaction and a genetic feedback loop, showing good agreement between the time-dependent distributions of molecule numbers predicted by the effective CME and CA simulations.

q-bio.QM↗

arXiv2022

Resilience of Bayesian Layer-Wise Explanations under Adversarial Attacks

We consider the problem of the stability of saliency-based explanations of Neural Network predictions under adversarial attacks in a classification task. Saliency interpretations of deterministic Neural Networks are remarkably brittle even when the attacks fail, i.e. for attacks that do not change the classification label. We empirically show that interpretations provided by Bayesian Neural Networks are considerably more stable under adversarial perturbations of the inputs and even under direct attacks to the explanations. By leveraging recent results, we also provide a theoretical explanation of this result in terms of the geometry of the data manifold. Additionally, we discuss the stability of the interpretations of high level representations of the inputs in the internal layers of a Network. Our results demonstrate that Bayesian methods, in addition to being more robust to adversarial attacks, have the potential to provide more stable and interpretable assessments of Neural Network predictions.

cs.LG↗

arXiv2021

Random Projections for Improved Adversarial Robustness

We propose two training techniques for improving the robustness of Neural Networks to adversarial attacks, i.e. manipulations of the inputs that are maliciously crafted to fool networks into incorrect predictions. Both methods are independent of the chosen attack and leverage random projections of the original inputs, with the purpose of exploiting both dimensionality reduction and some characteristic geometrical properties of adversarial perturbations. The first technique is called RP-Ensemble and consists of an ensemble of networks trained on multiple projected versions of the original inputs. The second one, named RP-Regularizer, adds instead a regularization term to the training objective.

cs.LG↗

arXiv2020

Systematic errors in estimates of $R_t$ from symptomatic cases in the presence of observation bias

We consider the problem of estimating the reproduction number $R_t$ of an epidemic for populations where the probability of detection of cases depends on a known covariate. We argue that in such cases the normal empirical estimator can fail when the prevalence of cases among groups changes with time. We propose a Bayesian strategy to resolve the problem, as well as a simple solution in the case of large number of cases. We illustrate the issue and its solution on a simple yet realistic simulation study, and discuss the general relevance of the issue to the current covid19 pandemic.

stat.AP↗