arXiv subjects

Paulo Yanez Sarmiento

Publications and source records attributed to Paulo Yanez Sarmiento.

4 recordsLinked to original sources

arXiv2026

Interpreting Protein Language Model Embeddings via Orthogonal Projection for Protein Fitness Prediction

Recently, there has been a growing adoption of protein language models (PLMs) in biomedical science. Their embeddings provide a rich numerical representation of protein sequences which achieve state-of-the-art performance on several downstream tasks including protein fitness prediction. However, PLM embeddings are not directly interpretable and, thereby, it remains unclear what features they encode. To gain insight into which biochemical properties of the protein are driving the prediction, we leverage an orthogonal projection technique that removes linear effects of known tabular features from embeddings and extend it to high-order and interaction effects. In this way, we remove the effects of interpretable biochemical features from PLM embeddings. In an ablation study, we show that this leads to a decrease in performance for a downstream classifier trained only on the embeddings to predict protein fitness. In an additional evaluation, we find that these biochemical features explain a substantial part of the variance in the predictions of this classifier. Hence, we can show that PLM embeddings encode patterns correlated with biochemical properties and quantify their contribution to predicting protein fitness. This computationally efficient approach is not limited to the features or embeddings considered here and is readily transferable to problem settings beyond protein fitness prediction.

cs.LG↗

arXiv2026

Evaluating Post-hoc Explanations of the Transformer-based Genome Language Model DNABERT-2

Explaining deep neural network predictions on genome sequences enables biological insight and hypothesis generation-often of greater interest than predictive performance alone. While explanations of convolutional neural networks (CNNs) have been shown to capture relevant patterns in genome sequences, it is unclear whether this transfers to more expressive Transformer-based genome language models (gLMs). To answer this question, we adapt AttnLRP, an extension of layer-wise relevance propagation to the attention mechanism, and apply it to the state-of-the-art gLM DNABERT-2. Thereby, we propose strategies to transfer explanations from token and nucleotide level. We evaluate the adaption of AttnLRP on genomic datasets using multiple metrics. Further, we provide an extensive comparison between the explanations of DNABERT-2 and a baseline CNN. Our results demonstrate that AttnLRP yields reliable explanations corresponding to known biological patterns. Hence, like CNNs, gLMs can also help derive biological insights. This work contributes to the explainability of gLMs and addresses the comparability of relevance attributions across different architectures.

cs.LG↗

arXiv2025

Sparse Explanations of Neural Networks Using Pruned Layer-Wise Relevance Propagation

Explainability is a key component in many applications involving deep neural networks (DNNs). However, current explanation methods for DNNs commonly leave it to the human observer to distinguish relevant explanations from spurious noise. This is not feasible anymore when going from easily human-accessible data such as images to more complex data such as genome sequences. To facilitate the accessibility of DNN outputs from such complex data and to increase explainability, we present a modification of the widely used explanation method layer-wise relevance propagation. Our approach enforces sparsity directly by pruning the relevance propagation for the different layers. Thereby, we achieve sparser relevance attributions for the input features as well as for the intermediate layers. As the relevance propagation is input-specific, we aim to prune the relevance propagation rather than the underlying model architecture. This allows to prune different neurons for different inputs and hence, might be more appropriate to the local nature of explanation methods. To demonstrate the efficacy of our method, we evaluate it on two types of data: images and genome sequences. We show that our modification indeed leads to noise reduction and concentrates relevance on the most important features compared to the baseline.

cs.LG↗

arXiv2021

Detection of arbitrage opportunities in multi-asset derivatives markets

We are interested in the existence of equivalent martingale measures and the detection of arbitrage opportunities in markets where several multi-asset derivatives are traded simultaneously. More specifically, we consider a financial market with multiple traded assets whose marginal risk-neutral distributions are known, and assume that several derivatives written on these assets are traded simultaneously. In this setting, there is a bijection between the existence of an equivalent martingale measure and the existence of a copula that couples these marginals. Using this bijection and recent results on improved Fréchet-Hoeffding bounds in the presence of additional information on functionals of a copula by Lux and Papapantoleon [18], we can extend the results of Tavin [33] on the detection of arbitrage opportunities to the general multi-dimensional case. More specifically, we derive sufficient conditions for the absence of arbitrage and formulate an optimization problem for the detection of a possible arbitrage opportunity. This problem can be solved efficiently using numerical optimization routines. The most interesting practical outcome is the following: we can construct a financial market where each multi-asset derivative is traded within its own no-arbitrage interval, and yet when considered together an arbitrage opportunity may arise.

q-fin.PR↗