arXiv subjects

J. Nathan Kutz

Publications and source records attributed to J. Nathan Kutz.

At least 235 records · Page 13Linked to original sources

arXiv2016

Modeling cognitive deficits following neurodegenerative diseases and traumatic brain injuries with deep convolutional neural networks

The accurate diagnosis and assessment of neurodegenerative disease and traumatic brain injuries (TBI) remain open challenges. Both cause cognitive and functional deficits due to focal axonal swellings (FAS), but it is difficult to deliver a prognosis due to our limited ability to assess damaged neurons at a cellular level in vivo. We simulate the effects of neurodegenerative disease and TBI using convolutional neural networks (CNNs) as our model of cognition. We utilize biophysically relevant statistical data on FAS to damage the connections in CNNs in a functionally relevant way. We incorporate energy constraints on the brain by pruning the CNNs to be less over-engineered. Qualitatively, we demonstrate that damage leads to human-like mistakes. Our experiments also provide quantitative assessments of how accuracy is affected by various types and levels of damage. The deficit resulting from a fixed amount of damage greatly depends on which connections are randomly injured, providing intuition for why it is difficult to predict impairments. There is a large degree of subjectivity when it comes to interpreting cognitive deficits from complex systems such as the human brain. However, we provide important insight and a quantitative framework for disorders in which FAS are implicated.

q-bio.NC↗

arXiv2016

Randomized Model Order Reduction

Singular value decomposition (SVD) has a crucial role in model order reduction. It is often utilized in the offline stage to compute basis functions that project the high-dimensional nonlinear problem into a low-dimensionsl model which is, then, evaluated cheaply. It constitutes a building block for many techniques such as e.g. Proper Orthogonal Decomposition and Dynamic Mode Decomposition. The aim of this work is to provide efficient computation of the basis functions via randomized matrix decompositions. This is possible due to the randomized Singular Value Decomposition (rSVD) which is a fast and accurate alternative of the SVD. Although this is considered as offline stage, this computation may be extremely expensive and therefore the use of compressed techniques drastically reduce its cost. Numerical examples show the effectiveness of the method for both Proper Orthogonal Decomposition (POD) and Dynamic Mode Decomposition (DMD).

math.NA↗

arXiv2016

Symmetries constrain dynamics in a family of balanced neural networks

We examine a family of random firing-rate neural networks in which we enforce the neurobiological constraint of Dale's Law --- each neuron makes either excitatory or inhibitory connections onto its post-synaptic targets. We find that this constrained system may be described as a perturbation from a system with non-trivial symmetries. We analyze the symmetric system using the tools of equivariant bifurcation theory, and demonstrate that the symmetry-implied structures remain evident in the perturbed system. In comparison, spectral characteristics of the network coupling matrix are relatively uninformative about the behavior of the constrained system.

q-bio.NC↗

arXiv2016

Associative Memory Impairments arising from Neurodegenerative Diseases and Traumatic Brain Injuries in a Hopfield Network Model

Neurodegenerative diseases and traumatic brain injuries (TBI) are among the main causes of cognitive dysfunction in humans. Both manifestations exhibit the extensive presence of focal axonal swellings (FAS). FAS compromises the information encoded in spike trains, thus leading to potentially severe functional deficits. Complicating our understanding of the impact of FAS is our inability to access small scale injuries with non-invasive methods, the overall complexity of neuronal pathologies, and our limited knowledge of how networks process biological signals. Building on Hopfield's pioneering work, we extend a model for associative memory to account for FAS and its impact on memory encoding. We calibrate all FAS parameters from biophysical observations of their statistical distribution and size, providing a framework to simulate the effects of brain disorders on memory recall performance. A face recognition example is used to demonstrate and validate the functionality of the novel model. Our results link memory recall ability to observed FAS statistics, allowing for a description of different stages of brain disorders within neuronal networks. This provides a first theoretical model to bridge experimental observations of FAS in neurodegeneration and TBI with compromised memory recall, thus closing the large gap between theory and experiment on how biological signals are processed in damaged, high-dimensional functional networks. The work further lends new insight into positing diagnostic tools to measure cognitive deficits.

q-bio.NC↗

arXiv2016

Data-driven discovery of partial differential equations

We propose a sparse regression method capable of discovering the governing partial differential equation(s) of a given system by time series measurements in the spatial domain. The regression framework relies on sparsity promoting techniques to select the nonlinear and partial derivative terms terms of the governing equations that most accurately represent the data, bypassing a combinatorially large search through all possible candidate models. The method balances model complexity and regression accuracy by selecting a parsimonious model via Pareto analysis. Time series measurements can be made in an Eulerian framework where the sensors are fixed spatially, or in a Lagrangian framework where the sensors move with the dynamics. The method is computationally efficient, robust, and demonstrated to work on a variety of canonical problems of mathematical physics including Navier-Stokes, the quantum harmonic oscillator, and the diffusion equation. Moreover, the method is capable of disambiguating between potentially non-unique dynamical terms by using multiple time series taken with different initial data. Thus for a traveling wave, the method can distinguish between a linear wave equation or the Korteweg-deVries equation, for instance. The method provides a promising new technique for discovering governing equations and physical laws in parametrized spatio-temporal systems where first-principles derivations are intractable.

nlin.PS↗

arXiv2016

Compressed Dynamic Mode Decomposition for Background Modeling

We introduce the method of compressed dynamic mode decomposition (cDMD) for background modeling. The dynamic mode decomposition (DMD) is a regression technique that integrates two of the leading data analysis methods in use today: Fourier transforms and singular value decomposition. Borrowing ideas from compressed sensing and matrix sketching, cDMD eases the computational workload of high resolution video processing. The key principal of cDMD is to obtain the decomposition on a (small) compressed matrix representation of the video feed. Hence, the cDMD algorithm scales with the intrinsic rank of the matrix, rather then the size of the actual video (data) matrix. Selection of the optimal modes characterizing the background is formulated as a sparsity-constrained sparse coding problem. Our results show, that the quality of the resulting background model is competitive, quantified by the F-measure, Recall and Precision. A GPU (graphics processing unit) accelerated implementation is also presented which further boosts the computational efficiency of the algorithm.

cs.CV↗

arXiv2016

Reaction time impairments in decision-making networks as a diagnostic marker for traumatic brain injuries and neurodegenerative diseases

The presence of diffuse Focal Axonal Swellings (FAS) is a hallmark cellular feature in many neurodegenerative diseases and traumatic brain injury. Among other things, the FAS have a significant impact on spike-train encodings that propagate through the affected neurons, leading to compromised signal processing on a neuronal network level. This work merges, for the first time, three fields of study: (i) signal processing in excitatory-inhibitory (EI) networks of neurons via population codes, (ii) decision-making theory driven by the production of evidence from stimulus, and (iii) compromised spike-train propagation through FAS. As such, we demonstrate a mathematical architecture capable of characterizing compromised decision-making driven by cellular mechanisms. The computational model also leads to several novel predictions and diagnostics for understanding injury level and cognitive deficits, including a key finding that decision-making reaction times, rather than accuracy, are indicative of network level damage. The results have a number of translational implications, including that the level of network damage can be characterized by the reaction times in simple cognitive and motor tests.

q-bio.NC↗

arXiv2016

Chaos as an Intermittently Forced Linear System

Understanding the interplay of order and disorder in chaotic systems is a central challenge in modern quantitative science. We present a universal, data-driven decomposition of chaos as an intermittently forced linear system. This work combines Takens' delay embedding with modern Koopman operator theory and sparse regression to obtain linear representations of strongly nonlinear dynamics. The result is a decomposition of chaotic dynamics into a linear model in the leading delay coordinates with forcing by low energy delay coordinates; we call this the Hankel alternative view of Koopman (HAVOK) analysis. This analysis is applied to the canonical Lorenz system, as well as to real-world examples such as the Earth's magnetic field reversal, and data from electrocardiogram, electroencephalogram, and measles outbreaks. In each case, the forcing statistics are non-Gaussian, with long tails corresponding to rare events that trigger intermittent switching and bursting phenomena; this forcing is highly predictive, providing a clear signature that precedes these events. Moreover, the activity of the forcing signal demarcates large coherent regions of phase space where the dynamics are approximately linear from those that are strongly nonlinear.

math.DS↗

arXiv2016

Shape Constrained Tensor Decompositions using Sparse Representations in Over-Complete Libraries

We consider $N$-way data arrays and low-rank tensor factorizations where the time mode is coded as a sparse linear combination of temporal elements from an over-complete library. Our method, Shape Constrained Tensor Decomposition (SCTD) is based upon the CANDECOMP/PARAFAC (CP) decomposition which produces $r$-rank approximations of data tensors via outer products of vectors in each dimension of the data. By constraining the vector in the temporal dimension to known analytic forms which are selected from a large set of candidate functions, more readily interpretable decompositions are achieved and analytic time dependencies discovered. The SCTD method circumvents traditional {\em flattening} techniques where an $N$-way array is reshaped into a matrix in order to perform a singular value decomposition. A clear advantage of the SCTD algorithm is its ability to extract transient and intermittent phenomena which is often difficult for SVD-based methods. We motivate the SCTD method using several intuitively appealing results before applying it on a number of high-dimensional, real-world data sets in order to illustrate the efficiency of the algorithm in extracting interpretable spatio-temporal modes. With the rise of data-driven discovery methods, the decomposition proposed provides a viable technique for analyzing multitudes of data in a more comprehensible fashion.

stat.ML↗

arXiv2016

Koopman Theory for Partial Differential Equations

We consider the application of Koopman theory to nonlinear partial differential equations. We demonstrate that the observables chosen for constructing the Koopman operator are critical for enabling an accurate approximation to the nonlinear dynamics. If such observables can be found, then the dynamic mode decomposition algorithm can be enacted to compute a finite-dimensional approximation of the Koopman operator, including its eigenfunctions, eigenvalues and Koopman modes. Judiciously chosen observables lead to physically interpretable spatio-temporal features of the complex system under consideration and provide a connection to manifold learning methods. We demonstrate the impact of observable selection, including kernel methods, and construction of the Koopman operator on two canonical, nonlinear PDEs: Burgers' equation and the nonlinear Schrödinger equation. These examples serve to highlight the most pressing and critical challenge of Koopman theory: a principled way to select appropriate observables.

nlin.PS↗

arXiv2016

Online interpolation point refinement for reduced order models using a genetic algorithm

A genetic algorithm procedure is demonstrated that refines the selection of interpolation points of the discrete empirical interpolation method (DEIM) when used for constructing reduced order models for time dependent and/or parametrized nonlinear partial differential equations (PDEs) with proper orthogonal decomposition. The method achieves nearly optimal interpolation points with only a few generations of the search, making it potentially useful for {\em online} refinement of the sparse sampling used to construct a projection of the nonlinear terms. With the genetic algorithm, points are optimized to jointly minimize reconstruction error and enable dynamic regime classification. The efficiency of the method is demonstrated on two canonical nonlinear PDEs: the cubic-quintic Ginzburg-Landau equation and the Navier-Stokes equation for flow around a cylinder. Using the former model, the procedure can be compared to the ground-truth optimal interpolation points, showing that the genetic algorithm quickly achieves nearly optimal performance and reduced the reconstruction error by nearly an order of magnitude.

math.NA↗

arXiv2016

Inferring biological networks by sparse identification of nonlinear dynamics

Inferring the structure and dynamics of network models is critical to understanding the functionality and control of complex systems, such as metabolic and regulatory biological networks. The increasing quality and quantity of experimental data enable statistical approaches based on information theory for model selection and goodness-of-fit metrics. We propose an alternative method to infer networked nonlinear dynamical systems by using sparsity-promoting $\ell_1$ optimization to select a subset of nonlinear interactions representing dynamics on a fully connected network. Our method generalizes the sparse identification of nonlinear dynamics (SINDy) algorithm to dynamical systems with rational function nonlinearities, such as biological networks. We show that dynamical systems with rational nonlinearities may be cast in an implicit form, where the equations may be identified in the null-space of a library of mixed nonlinearities including the state and derivative terms; this approach applies more generally to implicit dynamical systems beyond those containing rational nonlinearities. This method, implicit-SINDy, succeeds in inferring three canonical biological models: Michaelis-Menten enzyme kinetics, the regulatory network for competence in bacteria, and the metabolic network for yeast glycolysis.

math.DS↗

arXiv2016

Sparse Identification of Nonlinear Dynamics with Control (SINDYc)

Identifying governing equations from data is a critical step in the modeling and control of complex dynamical systems. Here, we investigate the data-driven identification of nonlinear dynamical systems with inputs and forcing using regression methods, including sparse regression. Specifically, we generalize the sparse identification of nonlinear dynamics (SINDY) algorithm to include external inputs and feedback control. This method is demonstrated on examples including the Lotka-Volterra predator--prey model and the Lorenz system with forcing and control. We also connect the present algorithm with the dynamic mode decomposition (DMD) and Koopman operator theory to provide a broader context.

math.DS↗

arXiv2016

Generalizing Koopman Theory to allow for inputs and control

We develop a new generalization of Koopman operator theory that incorporates the effects of inputs and control. Koopman spectral analysis is a theoretical tool for the analysis of nonlinear dynamical systems. Moreover, Koopman is intimately connected to Dynamic Mode Decomposition (DMD), a method that discovers spatial-temporal coherent modes from data, connects local-linear analysis to nonlinear operator theory, and importantly creates an equation-free architecture allowing investigation of complex systems. In actuated systems, standard Koopman analysis and DMD are incapable of producing input-output models; moreover, the dynamics and the modes will be corrupted by external forcing. Our new theoretical developments extend Koopman operator theory to allow for systems with nonlinear input-output characteristics. We show how this generalization is rigorously connected and generalizes a recent development called Dynamic Mode Decomposition with control (DMDc). We demonstrate this new theory on nonlinear dynamical systems, including a standard Susceptible-Infectious-Recovered model with relevance to the analysis of infectious disease data with mass vaccination (actuation).

math.OC↗

arXiv2016

Nonlinear model order reduction via Dynamic Mode Decomposition

We propose a new technique for obtaining reduced order models for nonlinear dynamical systems. Specifically, we advocate the use of the recently developed Dynamic Mode Decomposition (DMD), an equation-free method, to approximate the nonlinear term. DMD is a spatio-temporal matrix decomposition of a data matrix that correlates spatial features while simultaneously associating the activity with periodic temporal behavior. With this decomposition, one can obtain a fully reduced dimensional surrogate model and avoid the evaluation of the nonlinear term in the online stage. This allows for an impressive speed up of the computational cost, and, at the same time, accurate approximations of the problem. We present a suite of numerical tests to illustrate our approach and to show the effectiveness of the method in comparison to existing approaches.

math.NA↗

arXiv2015

Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control

In this work, we explore finite-dimensional linear representations of nonlinear dynamical systems by restricting the Koopman operator to an invariant subspace. The Koopman operator is an infinite-dimensional linear operator that evolves observable functions of the state-space of a dynamical system [Koopman 1931, PNAS]. Dominant terms in the Koopman expansion are typically computed using dynamic mode decomposition (DMD). DMD uses linear measurements of the state variables, and it has recently been shown that this may be too restrictive for nonlinear systems [Williams et al. 2015, JNLS]. Choosing nonlinear observable functions to form an invariant subspace where it is possible to obtain linear models, especially those that are useful for control, is an open challenge. Here, we investigate the choice of observable functions for Koopman analysis that enable the use of optimal linear control techniques on nonlinear problems. First, to include a cost on the state of the system, as in linear quadratic regulator (LQR) control, it is helpful to include these states in the observable subspace, as in DMD. However, we find that this is only possible when there is a single isolated fixed point, as systems with multiple fixed points or more complicated attractors are not globally topologically conjugate to a finite-dimensional linear system, and cannot be represented by a finite-dimensional linear Koopman subspace that includes the state. We then present a data-driven strategy to identify relevant observable functions for Koopman analysis using a new algorithm to determine terms in a dynamical system by sparse regression of the data in a nonlinear function space [Brunton et al. 2015, arxiv]; we show how this algorithm is related to DMD. Finally, we demonstrate how to design optimal control laws for nonlinear systems using techniques from linear optimal control on Koopman invariant subspaces.

math.DS↗

arXiv2015

Discovering governing equations from data: Sparse identification of nonlinear dynamical systems

The ability to discover physical laws and governing equations from data is one of humankind's greatest intellectual achievements. A quantitative understanding of dynamic constraints and balances in nature has facilitated rapid development of knowledge and enabled advanced technological achievements, including aircraft, combustion engines, satellites, and electrical power. In this work, we combine sparsity-promoting techniques and machine learning with nonlinear dynamical systems to discover governing physical equations from measurement data. The only assumption about the structure of the model is that there are only a few important terms that govern the dynamics, so that the equations are sparse in the space of possible functions; this assumption holds for many physical systems. In particular, we use sparse regression to determine the fewest terms in the dynamic governing equations required to accurately represent the data. The resulting models are parsimonious, balancing model complexity with descriptive ability while avoiding overfitting. We demonstrate the algorithm on a wide range of problems, from simple canonical systems, including linear and nonlinear oscillators and the chaotic Lorenz system, to the fluid vortex shedding behind an obstacle. The fluid example illustrates the ability of this method to discover the underlying dynamics of a system that took experts in the community nearly 30 years to resolve. We also show that this method generalizes to parameterized, time-varying, or externally forced systems.

math.DS↗

arXiv2015

The role of multistability and transient trajectories in networked dynamical systems: Connectomic dynamics of C. elegans and behavioral assays

The neural dynamics of the nematode C. elegans are experimentally low-dimensional and correspond to discrete behavioral states, where previous modeling work has found neural proxies for some of these states. Experimental results further suggest that dynamics may be understood as long-timescale transitions between multiple low-dimensional attractors. To identify multistable regimes of our model, we develop a method for systematic generation of bifurcation diagrams and their analysis in an interpretable low-dimensional subspace, showing the existence and nature of multistable input responses at a glance. Stimulation of the PLM neuron pair, experimentally associated with forward movement and shown in simulation to drive a limit cycle, defines our low-dimensional projection space. We then obtain bifurcation diagrams for single-neuron excitation over a range of amplitudes and which classify whether the dynamics in this projection space are associated with a limit cycle, fixed point, or multiple states. In the specific case of compound input into both the PLM pair and ASK pair we discover bistability of a limit cycle and a fixed point, with transitional timescales between different states being much longer than other timescales in the system. This suggests consistency of our model with the characterization of dynamics in neural systems as long-timescale transitions between discrete, low-dimensional attractors corresponding to behavioral states. Our methodology thus prescribes a method for identifying these states and transitions in response to arbitrary input.

q-bio.NC↗