SearcharxivSearch

arXiv subjects

Purushottam D. Dixit

Publications and source records attributed to Purushottam D. Dixit.

At least 19 recordsLinked to original sources

Discovering interpretable low-dimensional dynamics using maximum entropy

Models (i.e., governing equations) are fundamental to science and engineering. Advances in data acquisition now make it possible to extract interpretable, low dimensional descriptions from high dimensional observations. However, existing approaches sacrifice either interpretability for reconstruction accuracy or infer symbolic dynamics without relating latent coordinates to physically meaningful observables. Here we present Edwin (maximum entropy driven compression with interpretable nonlinear model discovery), a unified framework that simultaneously performs dimensionality reduction using the dynamic maximum entropy (DME) principle and discovers sparse symbolic models governing latent dynamics, as well as the coupling between learned features and external metadata. We validate Edwin on diverse simulated systems, including stochastic diffusion, the Ornstein-Uhlenbeck process, self assembling particles, spiking neural populations, and low rank recurrent neural networks, as well as on a noisy experimental time series of aggregating RNA-liposome complexes. Across all systems, Edwin recovers low dimensional symbolic models that are physically interpretable and generalize to unseen conditions. Together, these results establish Edwin as a powerful framework for inferring interpretable, low dimensional dynamics directly from high dimensional data.

q-bio.QM

Minimax entropy: The statistical physics of optimal models

When constructing models of the world, we aim for optimal compressions: models that include as few details as possible while remaining as accurate as possible. But which details -- or features measured in data -- should we choose to include in a model? Here, using the minimum description length principle, we show that the optimal features are the ones that produce the maximum entropy model with minimum entropy, thus yielding a minimax entropy principle. We review applications, which range from machine learning to optimal models of biological networks. Naive implementations, however, are limited to systems with small numbers of states and features. We therefore require new theoretical insights and computational techniques to construct optimal compressions of high-dimensional datasets arising in large-scale experiments.

q-bio.QM

TMI: Thermodynamic inference of data manifolds

The Gibbs-Boltzmann distribution offers a physically interpretable way to massively reduce the dimensionality of high dimensional probability distributions where the extensive variables are `features' and the intensive variables are `descriptors'. However, not all probability distributions can be modeled using the Gibbs-Boltzmann form. Here, we present TMI: TMI, {\bf T}hermodynamic {\bf M}anifold {\bf I}nference; a thermodynamic approach to approximate a collection of arbitrary distributions. TMI simultaneously learns from data intensive and extensive variables and achieves dimensionality reduction through a multiplicative, positive valued, and interpretable decomposition of the data. Importantly, the reduced dimensional space of intensive parameters is not homogeneous. The Gibbs-Boltzmann distribution defines an analytically tractable Riemannian metric on the space of intensive variables allowing us to calculate geodesics and volume elements. We discuss the applications of TMI with multiple real and artificial data sets. Possible extensions are discussed as well.

cond-mat.stat-mech

Introducing user-prescribed constraints in Markov chains for nonlinear dimensionality reduction

Stochastic kernel based dimensionality reduction approaches have become popular in the last decade. The central component of many of these methods is a symmetric kernel that quantifies the vicinity between pairs of data points and a kernel-induced Markov chain on the data. Typically, the Markov chain is fully specified by the kernel through row normalization. However, in many cases, it is desirable to impose user-specified stationary-state and dynamical constraints on the Markov chain. Unfortunately, no systematic framework exists to impose such user-defined constraints. Here, we introduce a path entropy maximization based approach to derive the transition probabilities of Markov chains using a kernel and additional user-specified constraints. We illustrate the usefulness of these Markov chains with examples.

cs.LG

Entropy production rate as a criterion for inconsistency in decision theory

Individual and group decisions are complex, often involving choosing an apt alternative from a multitude of options. Evaluating pairwise comparisons breaks down such complex decision problems into tractable ones. Pairwise comparison matrices (PCMs) are regularly used to solve multiple-criteria decision-making (MCDM) problems, for example, using Saaty's analytic hierarchy process (AHP) framework. However, there are two significant drawbacks of using PCMs. First, humans evaluate PCMs in an inconsistent manner. Second, not all entries of a large PCM can be reliably filled by human decision makers. We address these two issues by first establishing a novel connection between PCMs and time-irreversible Markov processes. Specifically, we show that every PCM induces a family of dissipative maximum path entropy random walks (MERW) over the set of alternatives. We show that only `consistent' PCMs correspond to detailed balanced MERWs. We identify the non-equilibrium entropy production in the induced MERWs as a metric of inconsistency of the underlying PCMs. Notably, the entropy production satisfies all of the recently laid out criteria for reasonable consistency indices. We also propose an approach to use incompletely filled PCMs in AHP. Potential future avenues are discussed as well. keywords: analytic hierarchy process, markov chains, maximum entropy

cs.AI

Maximum Caliber: a general variational principle for dynamical systems

We review here {\it Maximum Caliber} (Max Cal), a general variational principle for inferring distributions of paths in dynamical processes and networks. Max Cal is to dynamical trajectories what the principle of {\it Maximum Entropy} (Max Ent) is to equilibrium states or stationary populations. In Max Cal, you maximize a path entropy over all possible pathways, subject to dynamical constraints, in order to predict relative path weights. Many well-known relationships of Non-Equilibrium Statistical Physics -- such as the Green-Kubo fluctuation-dissipation relations, Onsager's reciprocal relations, and Prigogine's Minimum Entropy Production -- are limited to near-equilibrium processes. Max Cal is more general. While it can readily derive these results under those limits, Max Cal is also applicable far from equilibrium. We give recent examples of MaxCal as a method of inference about trajectory distributions from limited data, finding reaction coordinates in bio-molecular simulations, and modeling the complex dynamics of non-thermal systems such as gene regulatory networks or the collective firing of neurons. We also survey its basis in principle, and some limitations.

cond-mat.stat-mech

Caliber Corrected Markov Modeling (C2M2): Correcting Equilibrium Markov models

Rate processes are often modeled using Markov-State Models (MSM). Suppose you know a prior MSM, and then learn that your prediction of some particular observable rate is wrong. What is the best way to correct the whole MSM? For example, molecular dynamics simulations of protein folding may sample many microstates, possibly giving correct pathways through them, while also giving the wrong overall folding rate, when compared to experiment. Here, we describe Caliber Corrected Markov Modeling (C2M2): an approach based on the principle of maximum entropy for updating a Markov model by imposing state- and trajectory- based constraints. We show that such corrections are equivalent to asserting position-dependent diffusion coefficients in continuous-time continuous-space Markov processes modeled by a Smoluchowski equation. We derive the functional form of the diffusion coefficient explicitly in terms of the trajectory-based constraints. We illustrate with examples of 2D particle diffusion and an overdamped harmonic oscillator.

cond-mat.stat-mech

Mini-grand canonical ensemble: chemical potential in the solvation shell

Quantifying the statistics of occupancy of solvent molecules in the vicinity of solutes is central to our understanding of solvation phenomena. Number fluctuations in small `solvation shells' around solutes cannot be described within the macroscopic grand canonical framework using a single chemical potential that represents the solvent `bath'. In this communication, we hypothesize that molecular-sized observation volumes such as solvation shells are best described by coupling the solvation shell with a mixture of particle baths each with its own chemical potential. We confirm our hypotheses by studying the enhanced fluctuations in the occupancy statistics of hard sphere solvent particles around a distinguished hard sphere solute particle. Connections with established theories of solvation are also discussed.

cond-mat.stat-mech

Stationary properties of maximum entropy random walks

Maximum entropy (maxEnt) inference of state probabilities using state-dependent constraints is popular in the study of complex systems. In stochastic dynamical systems, the effect of state space topology and path-dependent constraints on the inferred state probabilities is unknown. To that end, we derive the transition probabilities and the stationary distribution of a maximum {\it path} entropy Markov process subject to state- and path-dependent constraints. The stationary distribution reflects a competition between path multiplicity and imposed constraints and is significantly different from the Boltzmann distribution. We illustrate our results with a particle diffusing on an energy landscape. Connections with the path integral approach to diffusion are discussed.

cond-mat.stat-mech

Maximum caliber is a general variational principle for nonequilibrium statistical mechanics

There has been interest in finding a general variational principle for non-equilibrium statistical mechanics. We give evidence that Maximum Caliber (Max Cal) is such a principle. Max Cal, a variant of Maximum Entropy, predicts dynamical distribution functions by maximizing a path entropy subject to dynamical constraints, such as average fluxes. We first show that Max Cal leads to standard near-equilibrium results -including the Green-Kubo relations, Onsager's reciprocal relations of coupled flows, and Prigogine's principle of minimum entropy production -in a way that is particularly simple. More importantly, because Max Cal does not require any notion of 'local equilibrium', or any notion of entropy dissipation, or even any restriction to material physics, it is more general than many traditional approaches. We develop some generalizations of the Onsager and Prigogine results that apply arbitrarily far from equilibrium. Max Cal is not limited to materials and fluids; it also applies, for example, to flows and trafficking on networks more broadly.

cond-mat.stat-mech

Inferring transition rates on networks with incomplete knowledge

Across many fields, a problem of interest is to predict the transition rates between nodes of a network, given limited stationary state and dynamical information. We give a solution using the principle of Maximum Caliber. We find the transition rate matrix by maximizing the path entropy of a random walker on the network constrained to reproducing a stationary distribution and a few dynamical averages. A main finding here is that when constrained only by the mean jump rate, the rate matrix is given by a square-root dependence of the rate, $ω_{ab} \propto \sqrt{p_b/p_a}$, on $p_a$ and $p_b$, the stationary state populations at nodes a and b. We give two examples of our approach. First, we show that this method correctly predicts the correlated rates in a biochemical network of two genes, where we know the exact results from prior simulation. Second, we show that it correctly predicts rates of peptide conformational transitions, when compared to molecular dynamics simulations. This method can be used to infer large numbers of rates on known networks where smaller numbers of steady-state node populations are known.

q-bio.MN

Detecting temperature fluctuations at equilibrium

Gibbs and Boltzmann definitions of temperature agree only in the macroscopic limit. The ambiguity in identifying the equilibrium temperature of a finite sized `small' system exchanging energy with a bath is usually understood as a limitation of conventional statistical mechanics. We interpret this ambiguity as resulting from a stochastically fluctuating temperature coupled with the phase space variables giving rise to a broad temperature distribution. With this ansatz, we develop the equilibrium statistics and dynamics of small systems. Numerical evidence using an analytically tractable model shows that the effects of temperature fluctuations can be detected in equilibrium and dynamical properties of the phase space of the small system. Our theory generalizes statistical mechanics to small systems relevant to biophysics and nanotechnology.

physics.bio-ph

A maximum entropy framework for non-exponential distributions

Probability distributions having power-law tails are observed in a broad range of social, economic, and biological systems. We describe here a potentially useful common framework. We derive distribution functions $\{p_k\}$ for situations in which a `joiner particle' $k$ pays some form of price to enter a `community' of size $k-1$, where costs are subject to economies-of-scale (EOS). Maximizing the Boltzmann-Gibbs-Shannon entropy subject to this energy-like constraint predicts a distribution having a power-law tail; it reduces to the Boltzmann distribution in the absence of EOS. We show that the predicted function gives excellent fits to 13 different distribution functions, ranging from friendship links in social networks, to protein-protein interactions, to the severity of terrorist attacks. This approach may give useful insights into when to expect power-law distributions in the natural and social sciences.

physics.soc-ph

Inferring microscopic kinetics of a Markov process using maximum caliber

We present a principled approach for estimating the matrix of microscopic rates among states of a Markov process, given only its stationary state population distribution and a single average global kinetic observable. We adapt Maximum Caliber, a variational principle in which a path entropy is maximized over the distribution of all the possible trajectories, subject to basic kinetic constraints and some average dynamical observables. We show that this approach leads, under appropriate conditions, to the continuous-time master equation and a Smoluchowski-like equation that is valid for both equilibrium and non-equilibrium stationary states. We illustrate the method by computing the solvation dynamics of water molecules from molecular dynamics trajectories.

cond-mat.stat-mech

A maximum entropy thermodynamics of small systems

We present a maximum entropy approach to analyze the internal dynamics of a small system in contact with a large bath e.g. a solute-solvent system. For the small solute, the fluctuations around the mean values of observables are not negligible and the probability distribution P(r) of the state space depends on the intricate details of the interaction of the solute with the solvent. Here, we employ a superstatistical approach: P(r) is expressed as a marginal distribution summed over the variation in β, the inverse temperature of the solute. The joint distribution P(β,r) is estimated by maximizing its entropy. We also calculate the first order system-size corrections to the canonical ensemble description of the state space. We test the development on a simple harmonic oscillator interacting with two baths with very different chemical identities viz. a) Lennard-Jones particles and b) water molecules. In both cases, our method captures the state space of the oscillator sufficiently well. Future directions and connections with traditional statistical mechanics are discussed.

cond-mat.stat-mech

Quantifying extrinsic noise in gene expression using the maximum entropy framework

We present a maximum entropy framework to separate intrinsic and extrinsic contributions to noisy gene expression solely from the profile of expression. We express the experimentally accessible probability distribution of the copy number of the gene product (mRNA or protein) by accounting for possible variations in extrinsic factors. The distribution of extrinsic factors is estimated using the maximum entropy principle. Our results show that extrinsic factors qualitatively and quantitatively affect the probability distribution of the gene product. We work out, in detail, the transcription of mRNA from a constitutively expressed promoter in {\it E. coli}. We suggest that the variation in extrinsic factors may account for the observed {\it wider than Poisson} distribution of mRNA copy numbers. We successfully test our framework on a numerical simulation of a simple gene expression scheme that accounts for the variation in extrinsic factors. We also make falsifiable predictions, some of which are tested on previous experiments in {\it E. coli} while others need verification. Application of the current framework to more complex situations is also discussed.

q-bio.SC

Ion-water clusters, bulk medium effects, and ion hydration

Thermochemistry of gas-phase ion-water clusters together with estimates of the hydration free energy of the clusters and the water ligands are used to calculate the hydration free energy of the ion. Often the hydration calculations use a continuum model of the solvent. The primitive quasichemical approximation to the quasichemical theory provides a transparent framework to anchor such efforts. Here we evaluate the approximations inherent in the primitive quasichemical approach and elucidate the different roles of the bulk medium. We find that the bulk medium can stabilize configurations of the cluster that are usually not observed in the gas phase, while also simultaneously lowering the excess chemical potential of the ion. This effect is more pronounced for soft ions. Since the coordination number that minimizes the excess chemical potential of the ion is identified as the optimal or most probable coordination number, for such soft ions, the optimum cluster size and the hydration thermodynamics obtained without account of the bulk medium on the ion-water clustering reaction can be different from those observed in simulations of the aqueous ion. The ideas presented in this work are expected to be relevant to experimental studies that translate thermochemistry of ion-water clusters to the thermodynamics of the hydrated ion and to evolving theoretical approaches that combine high-level calculations on clusters with coarse-grained models of the medium.

physics.bio-ph

An elastic-network based local molecular field analysis of zinc-finger proteins

We study two designed and one natural zinc-finger peptide each with the Cys2His2 (CCHH) type of metal binding motif. In the approach we have developed, we describe the role of the protein and solvent outside the Zn(II)-CCHH metal-residue cluster by a molecular field represented by generalized harmonic restraints. The strength of the field is adjusted to reproduce the binding energy distribution of the metal with the cluster obtained in a reference all-atom simulation with empirical potentials. The quadratic field allows us to investigate analytically the protein restraints on the binding site in terms of its eigenmodes. Examining these eigenmodes suggests, consistent with experimental observations, the importance of the first histidine (H) in the CCHH cluster in metal binding. Further, the eigenvalues corresponding to these modes also indicate that the designed proteins form a tighter complex with the metal. We find that the bulk protein and solvent response tends to destabilize metal-binding, emphasizing that the favorable energetics of metal-residue interaction is necessary to drive folding in this system. The representation of the bulk protein and solvent response by a local field allows us to perform Monte Carlo simulations of the metal-residue cluster using quantum-chemical approaches, here using a semi-empirical Hamiltonian. For configurations sampled from this simulation, we study the free energy of replacing Zn(II) with Fe(II), Co(II), and Ni(II) using density functional theory. The calculated selectivities are in fair agreement with experimental results.

physics.bio-ph