arXiv subjects

Michele Garibbo

Publications and source records attributed to Michele Garibbo.

4 recordsLinked to original sources

arXiv2026

Reinforcement Learning Guides Generative Protein Language Models

Protein engineering can optimize molecules for biotechnology and therapeutics, but navigating the high-dimensional sequence landscape remains challenging. Protein language models (pLMs) have shown to to generate functional proteins far from natural sequences, yet their outputs tend to reflect prevalent properties in training data, limiting discovery of rare properties such as high catalytic activity or thermostability. Here, we introduce ProtRL, a reinforcement learning framework for pLMs that iteratively updates model parameters to maximize externally defined reward functions. Across diverse design tasks, ProtRL shifts generation toward specified objectives while maintaining sequence diversity. We demonstrate the optimization of target folds, bounded and continuous fitness predictors, and multi-objective optimization in binder design. As a proof of concept, we applied ProtRL to experimental feedback for the engineering of epidermal growth factor receptor binders. Testing fewer than 100 designed variants across the experimental campaign, ProtRL-guided optimization provided a final round in which 16 of 22 variants bound EGFR. The best variant showed a dissociation constant of 5.5 nM, representing a nine-fold improvement over wild-type EGF and higher affinity than previously reported EGF variants identified through substantially larger screening campaigns. Our code and models are publicly available at github.com/AI4PDLab/ProtRL

q-bio.BM↗

arXiv2026

Steering Generative Models for Protein Design: Aligning and Conditioning Strategies

Generative artificial intelligence models learn probability distributions from data and produce novel samples that capture the salient properties of their training sets. Proteins are particularly attractive for such approaches given their abundant data and the versatility of their representations, ranging from sequences to structures and functions. This versatility has motivated the rapid development of generative models for protein design, enabling the generation of functional proteins and enzymes with unprecedented success. However, because these models mirror their training distribution, they tend to sample from its most probable modes, while low-probability regions, often encoding valuable properties, remain underexplored. To address this challenge, recent work has proposed strategies for steering generative models toward user-specified properties. In this review, we survey and categorize these strategies, distinguishing approaches that modify model parameters, such as reinforcement learning or supervised fine-tuning, from those that keep the model's parameters fixed, including conditional generation, retrieval-augmented strategies, Bayesian guidance, and tailored sampling methods. Together, these developments are beginning to enable the steering of generative models toward proteins with desired properties.

q-bio.BM↗

arXiv2023

Taylor TD-learning

Many reinforcement learning approaches rely on temporal-difference (TD) learning to learn a critic. However, TD-learning updates can be high variance. Here, we introduce a model-based RL framework, Taylor TD, which reduces this variance in continuous state-action settings. Taylor TD uses a first-order Taylor series expansion of TD updates. This expansion allows Taylor TD to analytically integrate over stochasticity in the action-choice, and some stochasticity in the state distribution for the initial state and action of each TD update. We include theoretical and empirical evidence that Taylor TD updates are indeed lower variance than standard TD updates. Additionally, we show Taylor TD has the same stable learning guarantees as standard TD-learning with linear function approximation under a reasonable assumption. Next, we combine Taylor TD with the TD3 algorithm, forming TaTD3. We show TaTD3 performs as well, if not better, than several state-of-the art model-free and model-based baseline algorithms on a set of standard benchmark tasks.

cs.LG↗

arXiv2022

What deep reinforcement learning tells us about human motor learning and vice-versa

Machine learning and specifically reinforcement learning (RL) has been extremely successful in helping us to understand neural decision making processes. However, RL's role in understanding other neural processes especially motor learning is much less well explored. To explore this connection, we investigated how recent deep RL methods correspond to the dominant motor learning framework in neuroscience, error-based learning. Error-based learning can be probed using a mirror reversal adaptation paradigm, where it produces distinctive qualitative predictions that are observed in humans. We therefore tested three major families of modern deep RL algorithm on a mirror reversal perturbation. Surprisingly, all of the algorithms failed to mimic human behaviour and indeed displayed qualitatively different behaviour from that predicted by error-based learning. To fill this gap, we introduce a novel deep RL algorithm: model-based deterministic policy gradients (MB-DPG). MB-DPG draws inspiration from error-based learning by explicitly relying on the observed outcome of actions. We show MB-DPG captures (human) error-based learning under mirror-reversal and rotational perturbation. Next, we demonstrate error-based learning in the form of MB-DPG learns faster than canonical model-free algorithms on complex arm-based reaching tasks, while being more robust to (forward) model misspecification than model-based RL. These findings highlight the gap between current deep RL methods and human motor adaptation and offer a route to closing this gap, facilitating future beneficial interaction between between the two fields.

q-bio.NC↗