arXiv subjects

Paul G. Constantine

Publications and source records attributed to Paul G. Constantine.

At least 19 recordsLinked to original sources

arXiv2020

A Lipschitz Matrix for Parameter Reduction in Computational Science

We introduce the Lipschitz matrix: a generalization of the scalar Lipschitz constant for functions with many inputs. Among the Lipschitz matrices compatible a particular function, we choose the smallest such matrix in the Frobenius norm to encode the structure of this function. The Lipschitz matrix then provides a function-dependent metric on the input space. Altering this metric to reflect a particular function improves the performance of many tasks in computational science. Compared to the Lipschitz constant, the Lipschitz matrix reduces the worst-case cost of approximation, integration, and optimization; if the Lipschitz matrix is low-rank, this cost no longer depends on the dimension of the input, but instead on the rank of the Lipschitz matrix defeating the curse of dimensionality. Both the Lipschitz constant and matrix define uncertainty away from point queries of the function and by using the Lipschitz matrix we can reduce uncertainty. If we build a minimax space-filling design of experiments in the Lipschitz matrix metric, we can further reduce this uncertainty. When the Lipschitz matrix is approximately low-rank, we can perform parameter reduction by constructing a ridge approximation whose active subspace is the span of the dominant eigenvectors of the Lipschitz matrix. In summary, the Lipschitz matrix provides a new tool for analyzing and performing parameter reduction in complex models arising in computational science.

math.NA↗

arXiv2019

Randomized Learning of the Second-Moment Matrix of a Smooth Function

Consider an open set $\mathbb{D}\subseteq\mathbb{R}^n$, equipped with a probability measure $μ$. An important characteristic of a smooth function $f:\mathbb{D}\rightarrow\mathbb{R}$ is its \emph{second-moment matrix} $Σ_μ:=\int \nabla f(x) \nabla f(x)^* μ(dx) \in\mathbb{R}^{n\times n}$, where $\nabla f(x)\in\mathbb{R}^n$ is the gradient of $f(\cdot)$ at $x\in\mathbb{D}$ and $*$ stands for transpose. For instance, the span of the leading $r$ eigenvectors of $Σ_μ$ forms an \emph{active subspace} of $f(\cdot)$, which contains the directions along which $f(\cdot)$ changes the most and is of particular interest in \emph{ridge approximation}. In this work, we propose a simple algorithm for estimating $Σ_μ$ from random point evaluations of $f(\cdot)$ \emph{without} imposing any structural assumptions on $Σ_μ$. Theoretical guarantees for this algorithm are established with the aid of the same technical tools that have proved valuable in the context of covariance matrix estimation from partial measurements.

cs.IT↗

arXiv2019

A Lanczos-Stieltjes method for one-dimensional ridge function approximation and integration

Many of the input-parameter-to-output-quantity-of-interest maps that arise in computational science admit a surprising low-dimensional structure, where the outputs vary primarily along a handful of directions in the high-dimensional input space. This type of structure is well modeled by a ridge function, which is a composition of a low-dimensional linear transformation with a nonlinear function. If the goal is to compute statistics of the output (e.g., as in uncertainty quantification or robust design) then one should exploit this low-dimensional structure, when present, to accelerate computations. We develop Gaussian quadrature and the associated polynomial approximation for one-dimensional ridge functions. The key elements of our method are (i) approximating the univariate density of the given linear combination of inputs by repeated convolutions and (ii) a Lanczos-Stieltjes method for constructing orthogonal polynomials and Gaussian quadrature.

math.NA↗

arXiv2018

Inverse regression for ridge recovery: A data-driven approach for parameter reduction in computer experiments

Parameter reduction can enable otherwise infeasible design and uncertainty studies with modern computational science models that contain several input parameters. In statistical regression, techniques for sufficient dimension reduction (SDR) use data to reduce the predictor dimension of a regression problem. A computational scientist hoping to use SDR for parameter reduction encounters a problem: a computer prediction is best represented by a deterministic function of the inputs, so data comprised of computer simulation queries fail to satisfy the SDR assumptions. To address this problem, we interpret SDR methods sliced inverse regression (SIR) and sliced average variance estimation (SAVE) as estimating the directions of a ridge function, which is a composition of a low-dimensional linear transformation with a nonlinear function. Within this interpretation, SIR and SAVE estimate matrices of integrals whose column spaces are contained in the ridge directions' span; we analyze and numerically verify convergence of these column spaces as the number of computer model queries increases. Moreover, we show example functions that are not ridge functions but whose inverse conditional moment matrices are low-rank. Consequently, the computational scientist should beware when using SIR and SAVE for parameter reduction, since SIR and SAVE may mistakenly suggest that truly important directions are unimportant.

math.NA↗

arXiv2018

Inverse regression for ridge recovery II: Numerics

We investigate the application of sufficient dimension reduction (SDR) to a noiseless data set derived from a deterministic function of several variables. In this context, SDR provides a framework for ridge recovery. In this second part, we explore the numerical subtleties associated with using two inverse regression methods---sliced inverse regression (SIR) and sliced average variance estimation (SAVE)---for ridge recovery. This includes a detailed numerical analysis of the eigenvalues of the resulting matrices and the subspaces spanned by their columns. After this analysis, we demonstrate the methods on several numerical test problems.

math.NA↗

arXiv2018

Data-driven polynomial ridge approximation using variable projection

Inexpensive surrogates are useful for reducing the cost of science and engineering studies involving large-scale, complex computational models with many input parameters. A ridge approximation is one class of surrogate that models a quantity of interest as a nonlinear function of a few linear combinations of the input parameters. When used in parameter studies (e.g., optimization or uncertainty quantification), ridge approximations allow the low dimensional structure to be exploited, reducing the effective dimension. We introduce a new, fast algorithm for constructing a ridge approximation where the nonlinear function is a polynomial. This polynomial ridge approximation is chosen to minimize least squared mismatch between the surrogate and the quantity of interest on a given set of inputs. Naively, this would require optimizing both the polynomial coefficients and the linear combination of weights; the latter of which define a low-dimensional subspace of the input space. However, given a fixed subspace the optimal polynomial can be found by solving a linear least-squares problem, and hence by using variable projection the polynomial can be implicitly found leaving an optimization problem over the subspace alone. We provide an algorithm that finds this polynomial ridge approximation by minimizing over the Grassmann manifold of low-dimensional subspaces using a Gauss-Newton method. We provide details of this optimization algorithm and demonstrate its performance on several numerical examples. Our Gauss-Newton method has superior theoretical guarantees and faster convergence than the alternating approach for polynomial ridge approximation earlier proposed by Constantine, Eftekhari, Hokanson, and Ward [https://doi.org/10.1016/j.cma.2017.07.038] that alternates between (i) optimizing the polynomial coefficients given the subspace and (ii) optimizing the subspace given the coefficients.

math.NA↗

arXiv2017

Exact active subspace Metropolis-Hastings, with applications to the Lorenz-96 system

We consider the application of active subspaces to inform a Metropolis-Hastings algorithm, thereby aggressively reducing the computational dimension of the sampling problem. We show that the original formulation, as proposed by Constantine, Kent, and Bui-Thanh (SIAM J. Sci. Comput., 38(5):A2779-A2805, 2016), possesses asymptotic bias. Using pseudo-marginal arguments, we develop an asymptotically unbiased variant. Our algorithm is applied to a synthetic multimodal target distribution as well as a Bayesian formulation of a parameter inference problem for a Lorenz-96 system.

stat.CO↗

arXiv2017

Gauss-Christoffel quadrature for inverse regression: applications to computer experiments

Sufficient dimension reduction (SDR) provides a framework for reducing the predictor space dimension in regression problems. We consider SDR in the context of deterministic functions of several variables such as those arising in computer experiments. In this context, SDR serves as a methodology for uncovering ridge structure in functions, and two primary algorithms for SDR---sliced inverse regression (SIR) and sliced average variance estimation (SAVE)---approximate matrices of integrals using a sliced mapping of the response. We interpret this sliced approach as a Riemann sum approximation of the particular integrals arising in each algorithm. We employ well-known tools from numerical analysis---namely, multivariate tensor product Gauss-Christoffel quadrature and orthogonal polynomials---to produce new algorithms that improve upon the Riemann sum-based numerical integration in SIR and SAVE. We call the new algorithms Lanczos-Stieltjes inverse regression (LSIR) and Lanczos-Stieltjes average variance estimation (LSAVE) due to their connection with Stieltjes' method---and Lanczos' related discretization---for generating a sequence of polynomials that are orthogonal to a given measure. We show that the quadrature-based approach approximates the desired integrals, and we study the behavior of LSIR and LSAVE with three numerical examples. As expected in high order numerical integration, the quadrature-based LSIR and LSAVE exhibit exponential convergence in the integral approximations compared to the first order convergence of the classical SIR and SAVE. The disadvantage of LSIR and LSAVE is that the underlying tensor product quadrature suffers from the curse of dimensionality---that is, the number of quadrature nodes grows exponentially with the input space dimension. Therefore, the proposed approach is most appropriate for deterministic functions with fewer than ten independent inputs.

math.NA↗

arXiv2017

Data-driven dimensional analysis: algorithms for unique and relevant dimensionless groups

Classical dimensional analysis has two limitations: (i) the computed dimensionless groups are not unique, and (ii) the analysis does not measure relative importance of the dimensionless groups. We propose two algorithms for estimating unique and relevant dimensionless groups assuming the experimenter can control the system's independent variables and evaluate the corresponding dependent variable; e.g., computer experiments provide such a setting. The first algorithm is based on a response surface constructed from a set of experiments. The second algorithm uses many experiments to estimate finite differences over a range of the independent variables. Both algorithms are semi-empirical because they use experimental data to complement the dimensional analysis. We derive the algorithms by combining classical semi-empirical modeling with active subspaces, which---given a probability density on the independent variables---yield unique and relevant dimensionless groups. The connection between active subspaces and dimensional analysis also reveals that all empirical models are ridge functions, which are functions that are constant along low-dimensional subspaces in its domain. We demonstrate the proposed algorithms on the well-studied example of viscous pipe flow---both turbulent and laminar cases. The results include a new set of two dimensionless groups for turbulent pipe flow that are ordered by relevance to the system; the precise notion of relevance is closely tied to the derivative based global sensitivity metric from Sobol' and Kucherenko.

math.NA↗

arXiv2017

A near-stationary subspace for ridge approximation

Response surfaces are common surrogates for expensive computer simulations in engineering analysis. However, the cost of fitting an accurate response surface increases exponentially as the number of model inputs increases, which leaves response surface construction intractable for high-dimensional, nonlinear models. We describe ridge approximation for fitting response surfaces in several variables. A ridge function is constant along several directions in its domain, so fitting occurs on the coordinates of a low-dimensional subspace of the input space. We review essential theory for ridge approximation---e.g., the best mean-squared approximation and an optimal low-dimensional subspace---and we prove that the gradient-based active subspace is near-stationary for the least-squares problem that defines an optimal subspace. Motivated by the theory, we propose a computational heuristic that uses an estimated active subspace as an initial guess for a ridge approximation fitting problem. We show a simple example where the heuristic fails, which reveals a type of function for which the proposed approach is inappropriate. We then propose a simple alternating heuristic for fitting a ridge function, and we demonstrate the effectiveness of the active subspace initial guess applied to an airfoil model of drag as a function of its 18 shape parameters.

math.NA↗

arXiv2017

Time-dependent global sensitivity analysis with active subspaces for a lithium ion battery model

Renewable energy researchers use computer simulation to aid the design of lithium ion storage devices. The underlying models contain several physical input parameters that affect model predictions. Effective design and analysis must understand the sensitivity of model predictions to changes in model parameters, but global sensitivity analyses become increasingly challenging as the number of input parameters increases. Active subspaces are part of an emerging set of tools for discovering and exploiting low-dimensional structures in the map from high-dimensional inputs to model outputs. We extend linear and quadratic model-based heuristic for active sub- space discovery to time-dependent processes and apply the resulting technique to a lithium ion battery model. The results reveal low-dimensional structure and sensitivity metrics that a designer may exploit to study the relationship between parameters and predictions.

physics.comp-ph↗

arXiv2017

Active subspaces of airfoil shape parameterizations

Design and optimization benefit from understanding the dependence of a quantity of interest (e.g., a design objective or constraint function) on the design variables. A low-dimensional active subspace, when present, identifies important directions in the space of design variables; perturbing a design along the active subspace associated with a particular quantity of interest changes that quantity more, on average, than perturbing the design orthogonally to the active subspace. This low-dimensional structure provides insights that characterize the dependence of quantities of interest on design variables. Airfoil design in a transonic flow field with a parameterized geometry is a popular test problem for design methodologies. We examine two particular airfoil shape parameterizations, PARSEC and CST, and study the active subspaces present in two common design quantities of interest, transonic lift and drag coefficients, under each shape parameterization. We mathematically relate the two parameterizations with a common polynomial series. The active subspaces enable low-dimensional approximations of lift and drag that relate to physical airfoil properties. In particular, we obtain and interpret a two-dimensional approximation of both transonic lift and drag, and we show how these approximation inform a multi-objective design problem.

math.NA↗

arXiv2016

Dimension reduction in MHD power generation models: dimensional analysis and active subspaces

Magnetohydrodynamics (MHD)---the study of electrically conducting fluids---can be harnessed to produce efficient, low-emissions power generation. Today, computational modeling assists engineers in studying candidate designs for such generators. However, these models are computationally expensive, so studying the effects of the model's many input parameters on output predictions is typically infeasible. We study two approaches for reducing the input dimension of the models: (i) classical dimensional analysis based on the inputs' units and (ii) active subspaces, which reveal low-dimensional subspaces in the space of inputs that affect the outputs the most. We also review the mathematical connection between the two approaches that leads to consistent application. The dimension reduction yields insights into the driving factors in the MHD power generation models. We study both the simplified Hartmann problem, which admits closed form expressions for the quantities of interest, and a large-scale computational model with adjoint capabilities that enable the derivative computations needed to estimate the active subspaces.

math.NA↗

arXiv2016

Global sensitivity metrics from active subspaces

Predictions from science and engineering models depend on several input parameters. Global sensitivity analysis quantifies the importance of each input parameter, which can lead to insight into the model and reduced computational cost; commonly used sensitivity metrics include Sobol' total sensitivity indices and derivative-based global sensitivity measures. Active subspaces are an emerging set of tools for identifying important directions in a model's input parameter space; these directions can be exploited to reduce the model's dimension enabling otherwise infeasible parameter studies. In this paper, we develop global sensitivity metrics called activity scores from the active subspace, which yield insight into the important model parameters. We mathematically relate the activity scores to established sensitivity metrics, and we discuss computational methods to estimate the activity scores. We show two numerical examples with algebraic functions taken from simplified engineering models. For each model, we analyze the active subspace and discuss how to exploit the low-dimensional structure. We then show that input rankings produced by the activity scores are consistent with rankings produced by the standard metrics.

math.NA↗

arXiv2016

Accelerating MCMC with active subspaces

The Markov chain Monte Carlo (MCMC) method is the computational workhorse for Bayesian inverse problems. However, MCMC struggles in high-dimensional parameter spaces, since its iterates must sequentially explore the high-dimensional space. This struggle is compounded in physical applications when the nonlinear forward model is computationally expensive. One approach to accelerate MCMC is to reduce the dimension of the state space. Active subspaces are part of an emerging set of tools for subspace-based dimension reduction. An active subspace in a given inverse problem indicates a separation between a low-dimensional subspace that is informed by the data and its orthogonal complement that is constrained by the prior. With this information, one can run the sequential MCMC on the active variables while sampling independently according to the prior on the inactive variables. However, this approach to increase efficiency may introduce bias. We provide a bound on the Hellinger distance between the true posterior and its active subspace- exploiting approximation. And we demonstrate the active subspace-accelerated MCMC on two computational examples: (i) a two-dimensional parameter space with a quadratic forward model and one-dimensional active subspace and (ii) a 100-dimensional parameter space with a PDE-based forward model and a two-dimensional active subspace.

math.NA↗

arXiv2016

Many physical laws are ridge functions

A ridge function is a function of several variables that is constant along certain directions in its domain. Using classical dimensional analysis, we show that many physical laws are ridge functions; this fact yields insight into the structure of physical laws and motivates further study into ridge functions and their properties. We also connect dimensional analysis to modern subspace-based techniques for dimension reduction, including active subspaces in deterministic approximation and sufficient dimension reduction in statistical regression.

math.NA↗

arXiv2016

A randomized algorithm for enumerating zonotope vertices

We propose a randomized algorithm for enumerating the vertices of a zonotope, which is a low-dimensional linear projection of a hypercube. The algorithm produces a pair of the zonotope's vertices by sampling a random linear combination of the zonotope generators, where the combination's weights are the signs of the product between the zonotope's generator matrix and random vectors with normally distributed entries. We study the probability of recovering particular vertices and relate it to the vertices' normal cones. This study shows that if we terminate the randomized algorithm before all vertices are recovered, then the convex hull of the resulting vertex set approximates the zonotope. In high dimensions, we expect the enumeration algorithm to be most appropriate as an approximation algorithm---particularly for cases when existing methods are not practical.

math.NA↗

arXiv2015

Computing Active Subspaces Efficiently with Gradient Sketching

Active subspaces are an emerging set of tools for identifying and exploiting the most important directions in the space of a computer simulation's input parameters; these directions depend on the simulation's quantity of interest, which we treat as a function from inputs to outputs. To identify a function's active subspace, one must compute the eigenpairs of a matrix derived from the function's gradient, which presents challenges when the gradient is not available as a subroutine. We numerically study two methods for estimating the necessary eigenpairs using only linear measurements of the function's gradient. In practice, these measurements can be estimated by finite differences using only two function evaluations, regardless of the dimension of the function's input space.

math.NA↗