arXiv subjects

Llorenç Espinosa-Portalés

Publications and source records attributed to Llorenç Espinosa-Portalés.

6 recordsLinked to original sources

arXiv2026

ProtModernBERT: Compute-Efficient Protein Language Modeling

Large protein language models provide a powerful framework for learning sequence-function relationships and guiding experimental design. However, architectural improvements are underexplored, especially in low-compute regimes of relevance for resource-constrained research laboratories. We show that the training cost for encoder-only transformer architectures can be reduced by leveraging modern architectural refinements, offering an encouraging performance / compute-efficiency trade-off. We introduce ProtModernBERT, a modernized BERT-style model pre-trained on amino acid sequences, and compare it with ProtBert, Amplify, and ESM2 across a set of supervised structural tasks -encompassing secondary structure prediction, disorder prediction, and subcellular localization- and ProteinGym's zero-shot functional benchmark. We find that ProtModernBERT matches or closely tracks the performance of Amplify-120M and ProtBert with up to 80-90\% reduction in pre-training FLOPs, and narrows the gap to ESM2 when controlling for pre-training. Our analysis using a cluster-based validation protocol for hyperparameter selection and randomly initialized models disentangles the contributions of architecture and data, showing that modern components provide stronger inductive biases that are more consequential than parameter count in low-compute regimes.

cs.LG↗

arXiv2026

Pareto-optimal quantum kernel selection for unsupervised anomaly detection on real malware beaconing data

Quantum kernel methods are leading candidates for a practical quantum advantage in machine learning, but assessing that potential requires two quantities usually reported separately: how well a kernel performs on the task, and how far its geometry departs from the classical kernels available for the same problem. We introduce a fully unsupervised, multi-objective protocol that optimises simultaneously the normalised pseudo discrepancy (NPD), a label-free proxy for anomaly detection quality, and the geometric difference (GD) to a tuned classical reference kernel, selecting models from the resulting Pareto front. We apply it to malware beaconing detection in real network traffic, using a one-class support vector machine with fidelity and projected quantum kernels over four data encodings, on simulators and on IQM's 20-qubit Garnet processor. NPD-guided selection alone finds a fidelity kernel that beats the tuned classical baseline, but with a geometric difference too small to certify the gain as quantum. Projected kernels reach far larger geometric differences; the Pareto-selected one only marginally exceeds the baseline (AUC $0.782$ versus $0.765$, $g_{C\to Q}\approx 89>\sqrt{N}$ relative to that reference kernel), still below the NPD-selected fidelity kernel ($0.840$).

quant-ph↗

arXiv2025

epiGPTope: A machine learning-based epitope generator and classifier

Epitopes are short antigenic peptide sequences which are recognized by antibodies or immune cell receptors. These are central to the development of immunotherapies, vaccines, and diagnostics. However, the rational design of synthetic epitope libraries is challenging due to the large combinatorial sequence space, $20^n$ combinations for linear epitopes of n amino acids, making screening and testing unfeasible, even with high throughput experimental techniques. In this study, we present a large language model, epiGPTope, pre-trained on protein data and specifically fine-tuned on linear epitopes, which for the first time can directly generate novel epitope-like sequences, which are found to possess statistical properties analogous to the ones of known epitopes. This generative approach can be used to prepare libraries of epitope candidate sequences. We further train statistical classifiers to predict whether an epitope sequence is of bacterial or viral origin, thus narrowing the candidate library and increasing the likelihood of identifying specific epitopes. We propose that such combination of generative and predictive models can be of assistance in epitope discovery. The approach uses only primary amino acid sequences of linear epitopes, bypassing the need for a geometric framework or hand-crafted features of the sequences. By developing a method to create biologically feasible sequences, we anticipate faster and more cost-effective generation and screening of synthetic epitopes, with relevant applications in the development of new biotechnologies.

cs.LG↗

arXiv2022

Real-space Bell inequalities in de Sitter

Bell-inequality violations reveal the presence of quantum correlations between two particles that have interacted and then separated. Their generalisation to quantum fields is necessary to study a number of field-theoretic setups, such as cosmological density fluctuations. In this work, we show how Bell operators can be constructed for quantum fields in real space, and for Gaussian states we compute their expectation value in terms of the field power spectra. We then apply our formalism to a scalar field in de-Sitter space-time. We find that, in spite of the tremendous production of entangled particles with opposite wave momenta on large scales, Bell inequalities are not violated in real space. The reason is that, when considering measurements of a field at two distinct locations in real space, one implicitly traces over the configuration of the field at every other location, leading to a mixed bipartite system. This "effective decoherence" effect is responsible for the erasure of quantum features, and casts some doubts on our ability to reveal the quantum origin of cosmological structures. We finally discuss these results in the light of quantum discord.

quant-ph↗

arXiv2021

Long-range enhanced mutual information from inflation

The quantum origin of cosmological primordial perturbations is a cornerstone framework in the interplay between gravity and quantum physics. In this paper we study the mutual information between two spatial regions in a radiation-dominated universe filled by a curvature perturbation field in a squeezed state. We find an enhancement with respect to the usual mutual information of the Minkowski vacuum due to momentum modes affected by particle production during inflation. This result supports our previous claim of the existence of long-range correlations between Primordial Black Holes (PBH) at formation during the radiation era.

gr-qc↗

arXiv2020

Entanglement entropy of Primordial Black Holes after inflation

In this paper we study the survival of entanglement of a scalar field state created during inflation. We find that there exist UV-finite subdominant contributions to the entanglement entropy per momentum mode that scale with the number of e-folds between horizon exit and the end of inflation, and depend on the logarithm of the radius of the entangling surface, which can be taken to be the horizon sphere. We argue that this entanglement entropy allows for the formation of entangled Primordial Black Holes (PBH). We find that the entropy arising from the entanglement between PBH is small compared with their Bekenstein entropy.

gr-qc↗