SearcharxivSearch

arXiv subjects

Steven A. Frank

Publications and source records attributed to Steven A. Frank.

At least 19 recordsLinked to original sources

Causal attribution by the chain rule: unifying natural selection, learning, economics, and other disciplines

Analysis often splits change into components. For example, how much of the observed variance is caused by genes or environment? In many cases, the split is ultimately made by the logic of the chain rule, which divides the difference of a product into two terms. Each term quantifies the partial difference associated with change in one component while holding the other component constant. The chain rule is of course widely known. However, this article argues that its deep fundamental role often goes unrecognized. The article shows how simply the basic chain rule unifies Fisher's fundamental theorem of natural selection, the Price equation description of evolutionary change, the Oaxaca-Blinder decomposition of wage differences in economics, the Kitagawa decomposition of mortality differences in demography, many expressions of thermodynamics, and most strikingly back propagation, the core optimization method of modern machine learning and artificial intelligence. The success in creating good designs and finding good solutions in both natural selection and artificial intelligence depends on how the chain rule propagates causes from instances of success or failure back to the underlying genes or parameters of the system. The mathematical analysis presented here shows that, for finite differences, the product rule form of the chain rule yields a basic decomposition of change into two components of a regression equation. That regression decomposition is purely a description of change with no explicit causal meaning. However, simple additional assumptions lead naturally to the modern counterfactual analysis of causality. From that perspective, we can easily understand the causal interpretation that Fisher gave to his fundamental theorem, and we can see the same causal structure in the Oaxaca-Blinder decomposition of economics and in causal analyses across many disciplines.

stat.ME

The Price equation reveals a universal force-metric-bias law of algorithmic learning and natural selection

Diverse learning algorithms, optimization methods, and natural selection share a common mathematical structure, despite their apparent differences. Here I show that a simple notational partitioning of change by the Price equation reveals a universal force-metric-bias (FMB) law: $Δ\mathbfθ = \mathbf{M}\,\mathbf{f} + \mathbf{b} + \mathbfξ$. The force $\mathbf{f}$ drives improvement in parameters, $Δ\mathbfθ$, in proportion to the slope of performance with respect to the parameters. The metric $\mathbf{M}$ rescales movement by inverse curvature. The bias $\mathbf{b}$ adds momentum or changes in the frame of reference. The noise $\mathbfξ$ enables exploration. This framework unifies natural selection, Bayesian updating, Newton's method, stochastic gradient descent, stochastic Langevin dynamics, Adam optimization, and most other algorithms as special cases of the same underlying process. The Price equation also reveals why Fisher information, Kullback-Leibler divergence, and d'Alembert's principle arise naturally in learning dynamics. By exposing this common structure, the FMB law provides a principled foundation for understanding, comparing, and designing learning algorithms across disciplines.

cs.LG

Circuit design in biology and machine learning. II. Anomaly detection

Anomaly detection is a well-established field in machine learning, identifying observations that deviate from typical patterns. The principles of anomaly detection could enhance our understanding of how biological systems recognize and respond to atypical environmental inputs. However, this approach has received limited attention in analyses of cellular and physiological circuits. This study builds on machine learning techniques -- such as dimensionality reduction, boosted decision trees, and anomaly classification -- to develop a conceptual framework for biological circuits. One problem is that machine learning circuits tend to be unrealistically large for use by cellular and physiological systems. I therefore focus on minimal circuits inspired by machine learning concepts, reduced to cellular scale. Through illustrative models, I demonstrate that small circuits can provide useful classification of anomalies. The analysis also shows how principles from machine learning -- such as temporal and atemporal anomaly detection, multivariate signal integration, and hierarchical decision-making cascades -- can inform hypotheses about the design and evolution of cellular circuits. This interdisciplinary approach enhances our understanding of cellular circuits and highlights the universal nature of computational strategies across biological and artificial systems.

q-bio.PE

Circuit design in biology and machine learning. I. Random networks and dimensional reduction

A biological circuit is a neural or biochemical cascade, taking inputs and producing outputs. How have biological circuits learned to solve environmental challenges over the history of life? The answer certainly follows Dobzhansky's famous quote that ``nothing in biology makes sense except in the light of evolution.'' But that quote leaves out the mechanistic basis by which natural selection's trial-and-error learning happens, which is exactly what we have to understand. How does the learning process that designs biological circuits actually work? How much insight can we gain about the form and function of biological circuits by studying the processes that have made those circuits? Because life's circuits must often solve the same problems as those faced by machine learning, such as environmental tracking, homeostatic control, dimensional reduction, or classification, we can begin by considering how machine learning designs computational circuits to solve problems. We can then ask: How much insight do those computational circuits provide about the design of biological circuits? How much does biology differ from computers in the particular circuit designs that it uses to solve problems? This article steps through two classic machine learning models to set the foundation for analyzing broad questions about the design of biological circuits. One insight is the surprising power of randomly connected networks. Another is the central role of internal models of the environment embedded within biological circuits, illustrated by a model of dimensional reduction and trend prediction. Overall, many challenges in biology have machine learning analogs, suggesting hypotheses about how biology's circuits are designed.

q-bio.PE

Natural selection at multiple scales

Natural selection acts on traits at different scales, often with opposing consequences. This article identifies the particular forces that act at each scale and how those forces combine to determine the overall evolutionary outcome. A series of extended models derive from the tragedy of the commons, illustrating opposing forces at different scales. Examples include the primary tension between conflict and cooperation and the evolution of virulence, sex ratios, dispersal, and evolvability. The unified analysis subsumes interactions within and between species by generalizing multitrait interactions. Expanded notions of recombination and cotransmission arise. The core theoretical approach isolates the fundamental forces of selection, including marginal valuation, correlation between interacting entities, and reproductive value. Those fundamental forces act as partial causes that combine at different temporal and spatial scales. Modeling focuses on statics, in the sense of how different forces at various scales tend to oppose each other, ultimately combining to shape traits. That type of static analysis emphasizes explanation rather than the calculation of dynamics. The article builds on the duality between explanation versus calculation in terms of statics versus dynamics. The literature often poses that duality as a controversy, whereas this article develops the pair as complementary tools that together provide deeper understanding. Along the way, the unified approach clarifies the subtle distinctions between kin selection, multilevel selection, and inclusive fitness, subsuming these topics into the broader perspectives of fundamental forces and multiple scales.

q-bio.PE

A biological circuit to anticipate trend

Organisms gain by anticipating future changes in the environment. Those environmental changes often follow stochastic trends. The greater the slope of the trend, the more likely the trend's momentum carries the future trend in the same direction. This article presents a simple biological circuit that measures the momentum, providing a prediction about future trend. The circuit calculates the momentum by the difference between a short-term and a long-term exponential moving average. The time lengths of the two moving averages can be adjusted by changing the decay rates of state variables. Different time lengths for those averages trade off between errors caused by noise and errors caused by lags in predicting a change in the direction of the trend. Prior studies have emphasized circuits that make similar calculations about trends. However, those prior studies embedded their analyses in the details of particular applications, obscuring the simple generality and wide applicability of the approach. The model here contributes to the topic by clarifying the great simplicity and generality of anticipation for stochastic trends. This article also notes that, in financial analysis, the difference between moving averages is widely used to predict future trends in asset prices. The financial measure is called the moving average convergence-divergence (MACD) indicator. Connecting the biological problem to financial analysis opens the way for future studies in biology to exploit the variety of highly developed trend models in finance.

q-bio.PE

Disease from opposing forces in regulatory control

Danger requires a strong rapid response. Speedy triggers are prone to false signals. False alarms can be costly, requiring strong negative regulators to oppose the initial triggers. Strongly opposed forces can easily be perturbed, leading to imbalance and disease. For example, immunity and fear response balance strong rapid triggers against widespread slow negative regulators. Diseases of immunity and behavior arise from imbalance. A different opposition of forces occurs in mammalian growth, which balances strong paternally expressed accelerators against maternally expressed suppressors. Diseases of overgrowth or undergrowth arise from imbalance. Other examples of opposing forces and disease include control of dopamine expression and male versus female favored traits.

q-bio.PE

Robustness and complexity

When a biological system robustly corrects component-level errors, the direct pressure on component performance declines. Components may become less reliable, maintain more genetic variability, or drift neutrally in design, creating the basis for new forms of organismal complexity. This article links the protection-decay dynamic to other aspects of robust and complex systems. Examples include the hourglass pattern of biological development and Doyle's hourglass architecture for robustly complex systems in engineering. The deeply and densely connected wiring architecture in biology's cellular controls and in machine learning's computational neural networks provide another link. By unifying these seemingly different aspects into a unified framework, we gain a new perspective on robust and complex systems.

q-bio.PE

Automatic differentiation and the optimization of differential equation models in biology

A computational revolution unleashed the power of artificial neural networks. At the heart of that revolution is automatic differentiation, which calculates the derivative of a performance measure relative to a large number of parameters. Differentiation enhances the discovery of improved performance in large models, an achievement that was previously difficult or impossible. Recently, a second computational advance optimizes the temporal trajectories traced by differential equations. Optimization requires differentiating a measure of performance over a trajectory, such as the closeness of tracking the environment, with respect to the parameters of the differential equations. Because model trajectories are usually calculated numerically by multistep algorithms, such as Runge-Kutta, the automatic differentiation must be passed through the numerical algorithm. This article explains how such automatic differentiation of trajectories is achieved. It also discusses why such computational breakthroughs are likely to advance theoretical and statistical studies of biological problems, in which one can consider variables as dynamic paths over time and space. Many common problems arise between improving success in computational learning models over performance landscapes, improving evolutionary fitness over adaptive landscapes, and improving statistical fits to data over information landscapes.

q-bio.QM

Optimizing differential equations to fit data and predict outcomes

Many scientific problems focus on observed patterns of change or on how to design a system to achieve particular dynamics. Those problems often require fitting differential equation models to target trajectories. Fitting such models can be difficult because each evaluation of the fit must calculate the distance between the model and target patterns at numerous points along a trajectory. The gradient of the fit with respect to the model parameters can be challenging. Recent technical advances in automatic differentiation through numerical differential equation solvers potentially change the fitting process into a relatively easy problem, opening up new possibilities to study dynamics. However, application of the new tools to real data may fail to achieve a good fit. This article illustrates how to overcome a variety of common challenges, using the classic ecological data for oscillations in hare and lynx populations. Models include simple ordinary differential equations (ODEs) and neural ordinary differential equations (NODEs), which use artificial neural networks to estimate the derivatives of differential equation systems. Comparing the fits obtained with ODEs versus NODEs, representing small and large parameter spaces, and changing the number of variable dimensions provide insight into the geometry of the observed and model trajectories. To analyze the quality of the models for predicting future observations, a Bayesian-inspired preconditioned stochastic gradient Langevin dynamics (pSGLD) calculation of the posterior distribution of predicted model trajectories clarifies the tendency for various models to underfit or overfit the data. Coupling fitted differential equation systems with pSGLD sampling provides a powerful way to study the properties of optimization surfaces, raising an analogy with mutation-selection dynamics on fitness landscapes.

q-bio.QM

The fundamental equations of change in statistical ensembles and biological populations

A recent article in Nature Physics unified key results from thermodynamics, statistics, and information theory. The unification arose from a general equation for the rate of change in the information content of a system. The general equation describes the change in the moments of an observable quantity over a probability distribution. One term in the equation describes the change in the probability distribution. The other term describes the change in the observable values for a given state. We show the equivalence of this general equation for moment dynamics with the widely known Price equation from evolutionary theory, named after George Price. We introduce the Price equation from its biological roots, review a mathematically abstract form of the equation, and discuss the potential for this equation to unify diverse mathematical theories from different disciplines. The new work in Nature Physics and many applications in biology show that this equation also provides the basis for deriving many novel theoretical results within each discipline.

q-bio.PE

Metabolic heat in microbial conflict and cooperation

Many microbes live in habitats below their optimum temperature. Retention of metabolic heat by aggregation or insulation would boost growth. Generation of excess metabolic heat may also provide benefit. A cell that makes excess metabolic heat pays the cost of production, whereas the benefit may be shared by neighbors within a zone of local heat capture. Metabolic heat as a shareable public good raises interesting questions about conflict and cooperation of heat production and capture. Metabolic heat may also be deployed as a weapon. Species with greater thermotolerance gain by raising local temperature to outcompete less thermotolerant taxa. Metabolic heat may provide defense against bacteriophage attack, by analogy with fever in vertebrates. This article outlines the theory of metabolic heat in microbial conflict and cooperation, presenting several predictions for future study.

q-bio.PE

The generalized Price equation: forces that change population statistics

The Price equation partitions the change in the expected value of a population measure. The first component describes the partial change caused by altered frequencies. The second component describes the partial change caused by altered measurements. In biology, frequency changes often associate with the direct effect of natural selection. Measure changes reflect processes during transmission that alter trait values. More broadly, the two components describe the direct forces that change population composition and the altered frame of reference that changes measured values. The classic Price equation is limited to population statistics that can expressed as the expected value of a measure. Many statistics cannot be expressed as expected values, such as the harmonic mean and the family of rescaled diversity measures. We generalize the Price equation to any population statistic that can be expressed as a function of frequencies and measurements. We obtain the generalized partition between the direct forces that cause frequency change and the altered frame of reference that changes measurements.

q-bio.PE

Invariance in ecological pattern

The abundance of different species in a community often follows the log series distribution. Other ecological patterns also have simple forms. Why does the complexity and variability of ecological systems reduce to such simplicity? Common answers include maximum entropy, neutrality, and convergent outcome from different underlying biological processes. This article proposes a more general answer based on the concept of invariance, the property by which a pattern remains the same after transformation. Invariance has a long tradition in physics. For example, general relativity emphasizes the need for the equations describing the laws of physics to have the same form in all frames of reference. By bringing this unifying invariance approach into ecology, we show that the log series pattern dominates when the consequences of processes acting on abundance are invariant to the addition or multiplication of abundance by a constant. The lognormal pattern dominates when the processes acting on net species growth rate obey rotational invariance (symmetry) with respect to the summing up of the individual component processes. Recognizing how these invariances connect pattern to process leads to a synthesis of previous approaches. First, invariance provides a simpler and more fundamental maximum entropy derivation of the log series distribution. Second, invariance provides a simple derivation of the key result from neutral theory: the log series at the metacommunity scale and a clearer form of the skewed lognormal at the local community scale. The invariance expressions are easy to understand because they uniquely describe the basic underlying components that shape pattern.

q-bio.PE

Simple unity among the fundamental equations of science

The Price equation describes the change in populations. Change concerns some value, such as biological fitness, information or physical work. The Price equation reveals universal aspects for the nature of change, independently of the meaning ascribed to values. By understanding those universal aspects, we can see more clearly why fundamental mathematical results in different disciplines often share a common form. We can also interpret more clearly the meaning of key results within each discipline. For example, the mathematics of natural selection in biology has a form closely related to information theory and physical entropy. Does that mean that natural selection is about information or entropy? Or do natural selection, information and entropy arise as interpretations of a common underlying abstraction? The Price equation suggests the latter. The Price equation achieves its abstract generality by partitioning change into two terms. The first term naturally associates with the direct forces that cause change. The second term naturally associates with the changing frame of reference. In the Price equation's canonical form, total change remains zero because the conservation of total probability requires that all probabilities invariantly sum to one. Much of the shared common form for the mathematics of different disciplines may arise from that seemingly trivial invariance of total probability, which leads to the partitioning of total change into equal and opposite components of the direct forces and the changing frame of reference.

q-bio.PE

The common patterns of abundance: the log series and Zipf's law

In a language corpus, the probability that a word occurs $n$ times is often proportional to $1/n^2$. Assigning rank, $s$, to words according to their abundance, $\log s$ vs $\log n$ typically has a slope of minus one. That simple Zipf's law pattern also arises in the population sizes of cities, the sizes of corporations, and other patterns of abundance. By contrast, for the abundances of different biological species, the probability of a population of size $n$ is typically proportional to $1/n$, declining exponentially for larger $n$, the log series pattern. This article shows that the differing patterns of Zipf's law and the log series arise as the opposing endpoints of a more general theory. The general theory follows from the generic form of all probability patterns as a consequence of conserved average values and the associated invariances of scale. To understand the common patterns of abundance, the generic form of probability distributions plus the conserved average abundance is sufficient. The general theory includes cases that are between the Zipf and log series endpoints, providing a broad framework for analyzing widely observed abundance patterns.

q-bio.PE

The Price equation program: simple invariances unify population dynamics, thermodynamics, probability, information and inference

The fundamental equations of various disciplines often seem to share the same basic structure. Natural selection increases information in the same way that Bayesian updating increases information. Thermodynamics and the forms of common probability distributions express maximum increase in entropy, which appears mathematically as loss of information. Physical mechanics follows paths of change that maximize Fisher information. The information expressions typically have analogous interpretations as the Newtonian balance between force and acceleration, representing a partition between direct causes of change and opposing changes in the frame of reference. This web of vague analogies hints at a deeper common mathematical structure. I suggest that the Price equation expresses that underlying universal structure. The abstract Price equation describes dynamics as the change between two sets. One component of dynamics expresses the change in the frequency of things, holding constant the values associated with things. The other component of dynamics expresses the change in the values of things, holding constant the frequency of things. The separation of frequency from value generalizes Shannon's separation of the frequency of symbols from the meaning of symbols in information theory. The Price equation's generalized separation of frequency and value reveals a few simple invariances that define universal geometric aspects of change. For example, the conservation of total frequency, although a trivial invariance by itself, creates a powerful constraint on the geometry of change. That constraint plus a few others seem to explain the common structural forms of the equations in different disciplines. From that abstract perspective, interpretations such as selection, information, entropy, force, acceleration, and physical work arise from the same underlying geometry expressed by the Price equation.

q-bio.PE

Receptor uptake arrays for vitamin B12, siderophores and glycans shape bacterial communities

Molecular variants of vitamin B12, siderophores and glycans occur. To take up variant forms, bacteria may express an array of receptors. The gut microbe Bacteroides thetaiotaomicron has three different receptors to take up variants of vitamin B12 and 88 receptors to take up various glycans. The design of receptor arrays reflects key processes that shape cellular evolution. Competition may focus each species on a subset of the available nutrient diversity. Some gut bacteria can take up only a narrow range of carbohydrates, whereas species such as B.~thetaiotaomicron can digest many different complex glycans. Comparison of different nutrients, habitats, and genomes provide opportunity to test hypotheses about the breadth of receptor arrays. Another important process concerns fluctuations in nutrient availability. Such fluctuations enhance the value of cellular sensors, which gain information about environmental availability and adjust receptor deployment. Bacteria often adjust receptor expression in response to fluctuations of particular carbohydrate food sources. Some species may adjust expression of uptake receptors for specific siderophores. How do cells use sensor information to control the response to fluctuations? That question about regulatory wiring relates to problems that arise in control theory and artificial intelligence. Control theory clarifies how to analyze environmental fluctuations in relation to the design of sensors and response systems. Recent advances in deep learning studies of artificial intelligence focus on the architecture of regulatory wiring and the ways in which complex control networks represent and classify environmental states. I emphasize the similar design problems that arise in cellular evolution, control theory, and artificial intelligence. I connect those broad concepts to testable hypotheses for bacterial uptake of B12, siderophores and glycans.

q-bio.MN