arXiv subjects

José L. Avalos

Publications and source records attributed to José L. Avalos.

5 recordsLinked to original sources

arXiv2025

Reinforcement learning for efficient and robust multi-setpoint and multi-trajectory tracking in bioprocesses

Efficient and robust bioprocess control is essential for maximizing performance and adaptability in advanced biotechnological systems. In this work, we present a reinforcement-learning framework for multi-setpoint and multi-trajectory tracking. Tracking multiple setpoints and time-varying trajectories in reinforcement learning is challenging due to the complexity of balancing multiple objectives, a difficulty further exacerbated by system uncertainties such as uncertain initial conditions and stochastic dynamics. This challenge is relevant, e.g., in bioprocesses involving microbial consortia, where precise control over population compositions is required. We introduce a novel return function based on multiplicative reciprocal saturation functions, which explicitly couples reward gains to the simultaneous satisfaction of multiple references. Through a case study involving light-mediated cybergenetic growth control in microbial consortia, we demonstrate via computational experiments that our approach achieves faster convergence, improved stability, and superior control compliance compared to conventional quadratic-cost-based return functions. Moreover, our method enables tuning of the saturation function's parameters, shaping the learning process and policy updates. By incorporating system uncertainties, our framework also demonstrates robustness, a key requirement in industrial bioprocessing. Overall, this work advances reinforcement-learning-based control strategies in bioprocess engineering, with implications in the broader field of process and systems engineering.

eess.SY↗

arXiv2025

Enhancing reinforcement learning for population setpoint tracking in co-cultures

Efficient multiple setpoint tracking can enable advanced biotechnological applications, such as maintaining desired population levels in co-cultures for optimal metabolic division of labor. In this study, we employ reinforcement learning as a control method for population setpoint tracking in co-cultures, focusing on policy-gradient techniques where the control policy is parameterized by neural networks. However, achieving accurate tracking across multiple setpoints is a significant challenge in reinforcement learning, as the agent must effectively balance the contributions of various setpoints to maximize the expected system performance. Traditional return functions, such as those based on a quadratic cost, often yield suboptimal performance due to their inability to efficiently guide the agent toward the simultaneous satisfaction of all setpoints. To overcome this, we propose a novel return function that rewards the simultaneous satisfaction of multiple setpoints and diminishes overall reward gains otherwise, accounting for both stage and terminal system performance. This return function includes parameters to fine-tune the desired smoothness and steepness of the learning process. We demonstrate our approach considering an $\textit{Escherichia coli}$ co-culture in a chemostat with optogenetic control over amino acid synthesis pathways, leveraging auxotrophies to modulate growth.

eess.SY↗

arXiv2024

Omics-driven hybrid dynamic modeling of bioprocesses with uncertainty estimation

This work presents an omics-driven modeling pipeline that integrates machine-learning tools to facilitate the dynamic modeling of multiscale biological systems. Random forests and permutation feature importance are proposed to mine omics datasets, guiding feature selection and dimensionality reduction for dynamic modeling. Continuous and differentiable machine-learning functions can be trained to link the reduced omics feature set to key components of the dynamic model, resulting in a hybrid model. As proof of concept, we apply this framework to a high-dimensional proteomics dataset of $\textit{Saccharomyces cerevisiae}$. After identifying key intracellular proteins that correlate with cell growth, targeted dynamic experiments are designed, and key model parameters are captured as functions of the selected proteins using Gaussian processes. This approach captures the dynamic behavior of yeast strains under varying proteome profiles while estimating the uncertainty in the hybrid model's predictions. The outlined modeling framework is adaptable to other scenarios, such as integrating additional layers of omics data for more advanced multiscale biological systems, or employing alternative machine-learning methods to handle larger datasets. Overall, this study outlines a strategy for leveraging omics data to inform multiscale dynamic modeling in systems biology and bioprocess engineering.

q-bio.QM↗

arXiv2024

Hybrid physics-informed metabolic cybergenetics: process rates augmented with machine-learning surrogates informed by flux balance analysis

Metabolic cybergenetics is a promising concept that interfaces gene expression and cellular metabolism with computers for real-time dynamic metabolic control. The focus is on control at the transcriptional level, serving as a means to modulate intracellular metabolic fluxes. Recent strategies in this field have employed constraint-based dynamic models for process optimization, control, and estimation. However, this results in bilevel dynamic optimization problems, which pose considerable numerical and conceptual challenges. In this study, we present an alternative hybrid physics-informed dynamic modeling framework for metabolic cybergenetics, aimed at simplifying optimization, control, and estimation tasks. By utilizing machine-learning surrogates, our approach effectively embeds the physics of metabolic networks into the process rates of structurally simpler macro-kinetic models coupled with gene expression. These surrogates, informed by flux balance analysis, link the domains of manipulatable intracellular enzymes to metabolic exchange fluxes. This ensures that critical knowledge captured by the system's metabolic network is preserved. The resulting models can be integrated into metabolic cybergenetic schemes involving single-level optimizations. Additionally, the hybrid modeling approach maintains the number of system states at a necessary minimum, easing the burden of process monitoring and estimation. Our hybrid physics-informed metabolic cybergenetic framework is demonstrated using a computational case study on the optogenetically-assisted production of itaconate by $\textit{Escherichia coli}$.

eess.SY↗

arXiv2024

Linking intra- and extra-cellular metabolic domains via neural-network surrogates for dynamic metabolic control

We outline a modeling and optimization strategy for investigating dynamic metabolic engineering interventions. Our framework is particularly useful at the early stages of research and development, often constrained by limited knowledge and experimental data. Elucidating a priori optimal trajectories of manipulatable intracellular fluxes can guide the design of suitable control schemes, e.g., cyber(ge)netic or in-cell approaches, and the selection of appropriate actuators, e.g., at the transcriptional or post-translational levels. Model-based dynamic optimization is proposed to predict optimal trajectories of target manipulatable intracellular fluxes. A challenge emerges as existing models are often oversimplified, lacking insights into metabolism, or excessively complex, making them difficult to build and implement. Here, we use surrogates derived from steady-state solutions of constraint-based metabolic models to link manipulatable intracellular fluxes to the process exchange rates of structurally simple hybrid dynamic models. The latter can be conveniently used in optimal control problems of metabolism. As a proof of concept, we apply our method to a reduced metabolic network of $\textit{Escherichia coli}$ considering two different scenarios of dynamic metabolic engineering.

eess.SY↗