A deep reinforcement learning model based on deterministic policy gradient for collective neural crest cell migration

Yihao Zhang; Zhaojie Chai; Yubing Sun; George Lykotrafitis

Home/Archive/arXiv

arXiv · 2007.03190

A deep reinforcement learning model based on deterministic policy gradient for collective neural crest cell migration

Yihao Zhang·Zhaojie Chai·Yubing Sun·George Lykotrafitis

View original source ↗Read source document ↗Export metadata ↓

Abstract

Modeling cell interactions such as co-attraction and contact-inhibition of locomotion is essential for understanding collective cell migration. Here, we propose a novel deep reinforcement learning model for collective neural crest cell migration. We apply the deep deterministic policy gradient algorithm in association with a particle dynamics simulation environment to train agents to determine the migration path. Because of the different migration mechanisms of leader and follower neural crest cells, we train two types of agents (leaders and followers) to learn the collective cell migration behavior. For a leader agent, we consider a linear combination of a global task, resulting in the shortest path to the target source, and a local task, resulting in a coordinated motion along the local chemoattractant gradient. For a follower agent, we consider only the local task. First, we show that the self-driven forces learned by the leader cell point approximately to the placode, which means that the agent is able to learn to follow the shortest path to the target. To validate our method, we compare the total time elapsed for agents to reach the placode computed using the proposed method and the time computed using an agent-based model. The distributions of the migration time intervals calculated using the two methods are shown to not differ significantly. We then study the effect of co-attraction and contact-inhibition of locomotion to the collective leader cell migration. We show that the overall leader cell migration for the case with co-attraction is slower because the co-attraction mitigates the source-driven effect. In addition, we find that the leader and follower agents learn to follow a similar migration behavior as in experimental observations. Overall, our proposed method provides useful insight on how to apply reinforcement learning techniques to simulate collective cell migration.

Explore related subjects

q-bio.CB

Keep this discovery

BibTeX RIS

Yihao Zhang, Zhaojie Chai, Yubing Sun, George Lykotrafitis. 2020-07-07. A deep reinforcement learning model based on deterministic policy gradient for collective neural crest cell migration. https://arxiv.org/abs/2007.03190

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Modeling Tissue Detachment and Rupture Using an Extended Vertex Model with T2-inverse Transitions

The vertex model is widely used to describe the mechanics of epithelial tissues, but its conventional formulation assumes that all cells remain tightly packed and always share edges with their neighbors, making it difficult to represent local detachment or gap formation. Here, we propose a minimal extension of the vertex model that enables cell detachment by introducing a new topological transformation, T2-inverse, which acts as the inverse of the classical T2 transition. When the imbalance of forces acting on a vertex, quantified by a tension metric $T$, exceeds a threshold, the T2-inverse splits the vertex into multiple vertices and creates a closed polygon that is incorporated as a pseudo-cell. This operation allows the model to represent the emergence and propagation of local detachment events. Using this framework, we simulate the stretching of a cell sheet and show that force-induced local detachments can accumulate to produce macroscopic tissue rupture. These results demonstrate that the proposed model extends the capability of the vertex model to describe tissue-level breakdown processes, including detachment and tearing, and provides a foundation for studying a broader class of epithelial mechanical phenomena.

q-bio.CB↗

arXiv2026

3D hybrid cellular Potts model with a discrete deformable fiber network: modeling cell contraction and extracellular matrix remodeling

The extracellular matrix (ECM) is a fibrous and dynamic network that plays a critical role in development, homeostasis, and disease. Cells both respond to and remodel the ECM, engaging in a mechanical reciprocity that shapes tissues. To study these interactions, computational models have been developed that simulate either ECM mechanics or cell behavior. The Cellular Potts Model (CPM) is a flexible cell-based framework that has been extended to many biological processes, including coupling to a discrete deformable fiber network. So far, however, this extension has only been applied in two dimensions, even though a three-dimensional setting is biologically more relevant. Here, we present a 3D hybrid framework that couples the CPM with a discrete and deformable representation of the ECM. This model enables explicit simulation of cell-induced ECM remodeling, including fiber reorientation and matrix densification. By incorporating contractile forces through static adhesion points, the model captures how ECM elasticity and fiber stiffness influences cell shape. The simulations show that ECM stiffness, controlled by fiber crosslinking, resists contraction and reaches equilibrium, while crosslink density modulates fiber alignment and local matrix accumulation. This framework provides a versatile platform for studying cell-ECM mechanics and supports future studies of multicellular behavior in realistic 3D environments.

q-bio.CB↗

arXiv2026

Coarse-Graining Agent-Based Models of Bacterial Infections

Agent-based models (ABMs) provide a natural framework for representing cell-level rules and spatial heterogeneity in bacterial infections, but their computational cost limits their use for macroscopic tissue-scale simulations and broad parameter exploration. We derive a deterministic coarse-grained description for a class of bacterial-infection ABMs in which immune cells and extracellular bacteria diffuse, immune cells ingest nearby bacteria, and intracellular bacterial loads evolve through prescribed birth and clearance processes. The first coarse-grained model is a semidiscrete reaction--diffusion system that retains a discrete internal state for each immune-cell bacterial load while representing cell and bacterial populations by continuum concentration fields. The key technical step is the derivation of state-dependent effective ingestion rates from the microscopic ABM parameters: these rates are obtained by solving an auxiliary diffusion problem around a single bacterium and computing the flux of immune cells into the interaction region. We then take a continuum limit in the internal state variable, yielding a state-structured reaction--diffusion system in which intracellular dynamics appear as advection and diffusion in state space. Numerical comparisons with ensemble-averaged ABM simulations show close agreement in biologically motivated parameter regimes. The resulting framework preserves the rule-based structure of the ABM while producing PDE models that are substantially more tractable for large-scale simulation and parameter studies.

q-bio.CB↗