arXiv subjects

Yannis Pantazis

Publications and source records attributed to Yannis Pantazis.

27 records · Page 2Linked to original sources

arXiv2018

Multiple Equivalent Solutions for the Lasso

Feature selection is an important problem studied in data analytics seeking to identify a minimal-size feature subset that is optimally predictive for an outcome of interest. It is also a powerful tool in Knowledge Discovery as a means for gaining domain insight, e.g., identifying which medical quantities carry unique information for the disease status. It is arguably less recognized however, that the problem may have multiple, equivalent solutions. In that case, it is misleading to domain experts to report only one of them and ignore all other equivalent solutions. In this paper, we extend a well-established single, feature selection algorithm (i.e., reporting a single solution), namely the Lasso algorithm, to the multiple solution problem based on formalized notion of equivalence for both classification and regression tasks. Empirical results are obtained using a fully automated pipeline called Just Add Data Bio or JAD Bio training and selecting multiple, linear as well as nonlinear learners, optimizing hyper-parameter values, and correcting for the bias of multiple inductions (model selection). The results show that multiple solutions do exist in real datasets, as well as the ability of the algorithm to identify a subset of them. A comparison with the Statistical Equivalent Solutions (SES) algorithm shows that Lasso equivalent solutions have better prediction performance at the cost of selecting more features.

math.ST↗

arXiv2015

Path-space information bounds for uncertainty quantification and sensitivity analysis of stochastic dynamics

Uncertainty quantification is a primary challenge for reliable modeling and simulation of complex stochastic dynamics. Such problems are typically plagued with incomplete information that may enter as uncertainty in the model parameters, or even in the model itself. Furthermore, due to their dynamic nature, we need to assess the impact of these uncertainties on the transient and long-time behavior of the stochastic models and derive corresponding uncertainty bounds for observables of interest. A special class of such challenges is parametric uncertainties in the model and in particular sensitivity analysis along with the corresponding sensitivity bounds for stochastic dynamics. Moreover, sensitivity analysis can be further complicated in models with a high number of parameters that render straightforward approaches, such as gradient methods, impractical. In this paper, we derive uncertainty and sensitivity bounds for path-space observables of stochastic dynamics in terms of new goal-oriented divergences; the latter incorporate both observables and information theory objects such as the relative entropy rate. These bounds are tight, depend on the variance of the particular observable and are computable through Monte Carlo simulation. In the case of sensitivity analysis, the derived sensitivity bounds rely on the path Fisher Information Matrix, hence they depend only on local dynamics and are gradient-free. These features allow for computationally efficient implementation in systems with a high number of parameters, e.g., complex reaction networks and molecular simulations.

math.PR↗

arXiv2015

Pathwise Sensitivity Analysis in Transient Regimes

The instantaneous relative entropy (IRE) and the corresponding instanta- neous Fisher information matrix (IFIM) for transient stochastic processes are pre- sented in this paper. These novel tools for sensitivity analysis of stochastic models serve as an extension of the well known relative entropy rate (RER) and the corre- sponding Fisher information matrix (FIM) that apply to stationary processes. Three cases are studied here, discrete-time Markov chains, continuous-time Markov chains and stochastic differential equations. A biological reaction network is presented as a demonstration numerical example.

math.PR↗

arXiv2014

Parametric Sensitivity Analysis for Stochastic Molecular Systems using Information Theoretic Metrics

In this paper we extend the parametric sensitivity analysis (SA) methodology proposed in Ref. [Y. Pantazis and M. A. Katsoulakis, J. Chem. Phys. 138, 054115 (2013)] to continuous time and continuous space Markov processes represented by stochastic differential equations and, particularly, stochastic molecular dynamics as described by the Langevin equation. The utilized SA method is based on the computation of the information-theoretic (and thermodynamic) quantity of relative entropy rate (RER) and the associated Fisher information matrix (FIM) between path distributions. A major advantage of the pathwise SA method is that both RER and pathwise FIM depend only on averages of the force field therefore they are tractable and computable as ergodic averages from a single run of the molecular dynamics simulation both in equilibrium and in non-equilibrium steady state regimes. We validate the performance of the extended SA method to two different molecular stochastic systems, a standard Lennard-Jones fluid and an all-atom methane liquid and compare the obtained parameter sensitivities with parameter sensitivities on three popular and well-studied observable functions, namely, the radial distribution function, the mean squared displacement and the pressure. Results show that the RER-based sensitivities are highly correlated with the observable-based sensitivities.

cs.IT↗

arXiv2014

Accelerated Sensitivity Analysis in High-Dimensional Stochastic Reaction Networks

In this paper, a two-step strategy for parametric sensitivity analysis for such systems is proposed, exploiting advantages and synergies between two recently proposed sensitivity analysis methodologies for stochastic dynamics. The first method performs sensitivity analysis of the stochastic dynamics by means of the Fisher Information Matrix on the underlying distribution of the trajectories; the second method is a reduced-variance, finite-difference, gradient-type sensitivity approach relying on stochastic coupling techniques for variance reduction. Here we demonstrate that these two methods can be combined and deployed together by means of a new sensitivity bound which incorporates the variance of the quantity of interest as well as the Fisher Information Matrix estimated from the first method. The first step of the proposed strategy labels sensitivities using the bound and screens out the insensitive parameters in a controlled manner based also on the new sensitivity bound. In the second step of the proposed strategy, the finite-difference method is applied only for the sensitivity estimation of the (potentially) sensitive parameters that have not been screened out in the first step. Results on an epidermal growth factor network with fifty parameters and on a protein homeostasis with eighty parameters demonstrate that the proposed strategy is able to quickly discover and discard the insensitive parameters and in the remaining potentially sensitive parameters it accurately estimates the sensitivities. The new sensitivity strategy can be several times faster than current state-of-the-art approaches that test all parameters, especially in "sloppy" systems. In particular, the computational acceleration is quantified by the ratio between the total number of parameters over the number of the sensitive parameters.

q-bio.MN↗

arXiv2013

Controlled-Error Approximations for Surface Diffusion of Interacting Particles with Applications to Pattern Formation

Microscopic processes on surfaces such as adsorption, desorption, diffusion and reaction of interacting particles can be simulated using kinetic Monte Carlo (kMC) algorithms. Even though kMC methods are accurate, they are computationally expensive for large-scale systems. Hence approximation algorithms are necessary for simulating experimentally observed properties and morphologies. One such approximation method stems from the coarse graining of the lattice which leads to coarse-grained Monte Carlo (GCMC) methods while Langevin approximations can further accelerate the simulations. Moreover, sacrificing fine scale (i.e. microscopic) accuracy, mesoscopic deterministic or stochastic partial differential equations (SPDEs) are efficiently applied for simulating surface processes. In this paper, we are interested in simulating surface diffusion for pattern formation applications which is achieved by suitably discretizing the mesoscopic SPDE in space. The proposed discretization schemes which are actually Langevin-type approximation models are strongly connected with the properties of the underlying interacting particle system. In this direction, the key feature of our schemes is that controlled-error estimates are provided at three distinct time-scales. Indeed, (a) weak error analysis of mesoscopic observables, (b) asymptotic equivalence of action functionals and (c) satisfaction of detailed balance condition, control the error at finite times, long times and infinite times, respectively. In this sense, the proposed algorithms provide a "bridge" between continuum (S)PDE models and molecular simulations Numerical simulations, which also take advantage of acceleration ideas from (S)PDE numerical solutions, validate the theoretical findings and provide insights to the experimentally observed pattern formation through self-assembly.

math-ph↗

arXiv2013

Parametric Sensitivity Analysis for Biochemical Reaction Networks based on Pathwise Information Theory

Stochastic modeling and simulation provide powerful predictive methods for the intrinsic understanding of fundamental mechanisms in complex biochemical networks. Typically, such mathematical models involve networks of coupled jump stochastic processes with a large number of parameters that need to be suitably calibrated against experimental data. In this direction, the parameter sensitivity analysis of reaction networks is an essential mathematical and computational tool, yielding information regarding the robustness and the identifiability of model parameters. However, existing sensitivity analysis approaches such as variants of the finite difference method can have an overwhelming computational cost in models with a high-dimensional parameter space. We develop a sensitivity analysis methodology suitable for complex stochastic reaction networks with a large number of parameters. The proposed approach is based on Information Theory methods and relies on the quantification of information loss due to parameter perturbations between time-series distributions. For this reason, we need to work on path-space, i.e., the set consisting of all stochastic trajectories, hence the proposed approach is referred to as "pathwise". The pathwise sensitivity analysis method is realized by employing the rigorously-derived Relative Entropy Rate (RER), which is directly computable from the propensity functions. A key aspect of the method is that an associated pathwise Fisher Information Matrix (FIM) is defined, which in turn constitutes a gradient-free approach to quantifying parameter sensitivities. The structure of the FIM turns out to be block-diagonal, revealing hidden parameter dependencies and sensitivities in reaction networks.

cs.IT↗

arXiv2013

Measuring the Irreversibility of Numerical Schemes for Reversible Stochastic Differential Equations

For a Markov process the detailed balance condition is equivalent to the time-reversibility of the process. For stochastic differential equations (SDE's) time discretization numerical schemes usually destroy the property of time-reversibility. Despite an extensive literature on the numerical analysis for SDE's, their stability properties, strong and/or weak error estimates, large deviations and infinite-time estimates, no quantitative results are known on the lack of reversibility of the discrete-time approximation process. In this paper we provide such quantitative estimates by using the concept of entropy production rate, inspired by ideas from non-equilibrium statistical mechanics. The entropy production rate for a stochastic process is defined as the relative entropy (per unit time) of the path measure of the process with respect to the path measure of the time-reversed process. By construction the entropy production rate is nonnegative and it vanishes if and only if the process is reversible. Crucially, from a numerical point of view, the entropy production rate is an {\em a posteriori} quantity, hence it can be computed in the course of a simulation as the ergodic average of a certain functional of the process (the so-called Gallavotti-Cohen (GC) action functional). We compute the entropy production for various numerical schemes such as explicit Euler-Maruyama and explicit Milstein's for reversible SDEs with additive or multiplicative noise. Additionally, we analyze the entropy production for the BBK integrator of the Langevin processes. We show that entropy production is an observable that distinguishes between different numerical schemes in terms of their discretization-induced irreversibility. Furthermore, our results show that the type of the noise critically affects the behavior of the entropy production rate.

math.NA↗

arXiv2012

A Relative Entropy Rate Method for Path Space Sensitivity Analysis of Stationary Complex Stochastic Dynamics

We propose a new sensitivity analysis methodology for complex stochastic dynamics based on the Relative Entropy Rate. The method becomes computationally feasible at the stationary regime of the process and involves the calculation of suitable observables in path space for the Relative Entropy Rate and the corresponding Fisher Information Matrix. The stationary regime is crucial for stochastic dynamics and here allows us to address the sensitivity analysis of complex systems, including examples of processes with complex landscapes that exhibit metastability, non-reversible systems from a statistical mechanics perspective, and high-dimensional, spatially distributed models. All these systems exhibit, typically non-gaussian stationary probability distributions, while in the case of high-dimensionality, histograms are impossible to construct directly. Our proposed methods bypass these challenges relying on the direct Monte Carlo simulation of rigorously derived observables for the Relative Entropy Rate and Fisher Information in path space rather than on the stationary probability distribution itself. We demonstrate the capabilities of the proposed methodology by focusing here on two classes of problems: (a) Langevin particle systems with either reversible (gradient) or non-reversible (non-gradient) forcing, highlighting the ability of the method to carry out sensitivity analysis in non-equilibrium systems; and, (b) spatially extended Kinetic Monte Carlo models, showing that the method can handle high-dimensional problems.

math-ph↗