Searcharxiv⌕ Search

arXiv subjects

Alex Townsend

Publications and source records attributed to Alex Townsend.

At least 73 records · Page 4Linked to original sources

A sparse spectral method on triangles

In this paper, we demonstrate that many of the computational tools for univariate orthogonal polynomials have analogues for a family of bivariate orthogonal polynomials on the triangle, including Clenshaw's algorithm and sparse differentiation operators. This allows us to derive a practical spectral method for solving linear partial differential equations on triangles with sparse discretizations. We can thereby rapidly solve partial differential equations using polynomials with degrees in the thousands, resulting in sparse discretizations with as many as several million degrees of freedom.

math.NA↗

Chebyshev approximation and the global geometry of sloppy models

Sloppy models are complex nonlinear models with outcomes that are significantly affected by only a small subset of parameter combinations. Despite forming an important universality class and arising frequently in practice, formal and systematic explanations of sloppiness are lacking. By unifying geometric interpretations of sloppiness with Chebyshev approximation theory, we offer such an explanation, and show how sloppiness can be described explicitly in terms of model smoothness. Our approach results in universal bounds on model predictions for classes of smooth models, and our bounds capture global geometric features that are intrinsic to their model manifolds. We illustrate these ideas using three disparate models: exponential decay, reaction rates from an enzyme-catalysed chemical reaction, and an epidemiology model of an infected population.

math.NA↗

Stokes' second problem and reduction of inertia in active fluids

We study a generalized Navier-Stokes model describing the thin-film flows in non-dilute suspensions of ATP-driven microtubules or swimming bacteria that are enclosed by a moving ring-shaped container. Considering Stokes' second problem, which concerns the motion of an oscillating boundary, our numerical analysis predicts that a periodically rotating ring will oscillate at a higher frequency in an active fluid than in a passive fluid, due to an activity-induced reduction of the fluid inertia. In the case of a freely suspended fluid-container system that is isolated from external forces or torques, active fluid stresses can induce large fluctuations in the container's angular momentum if the confinement radius matches certain multiples of the intrinsic vortex size of the active suspension. This effect could be utilized to transform collective microscopic swimmer activity into macroscopic motion in optimally tuned geometries.

cond-mat.soft↗

Why are Big Data Matrices Approximately Low Rank?

Matrices of (approximate) low rank are pervasive in data science, appearing in recommender systems, movie preferences, topic models, medical records, and genomics. While there is a vast literature on how to exploit low rank structure in these datasets, there is less attention on explaining why the low rank structure appears in the first place. Here, we explain the effectiveness of low rank models in data science by considering a simple generative model for these matrices: we suppose that each row or column is associated to a (possibly high dimensional) bounded latent variable, and entries of the matrix are generated by applying a piecewise analytic function to these latent variables. These matrices are in general full rank. However, we show that we can approximate every entry of an $m \times n$ matrix drawn from this model to within a fixed absolute error by a low rank matrix whose rank grows as $\mathcal O(\log(m + n))$. Hence any sufficiently large matrix from such a latent variable model can be approximated, up to a small entrywise error, by a low rank matrix.

cs.LG↗

Continuous analogues of Krylov methods for differential operators

Analogues of the conjugate gradient method, MINRES, and GMRES are derived for solving boundary value problems (BVPs) involving second-order differential operators. Two challenges arise: imposing the boundary conditions on the solution while building up a Krylov subspace, and guaranteeing convergence of the Krylov-based method on unbounded operators. Our approach employs projection operators to guarantee that the boundary conditions are satisfied, and we develop an operator preconditioner that ensures that an approximate solution is computed after a finite number of iterations. The developed Krylov methods are practical iterative BVP solvers that are particularly efficient when a fast operator-function product is available.

math.NA↗

Recurrence relations for orthogonal polynomials on a triangle

This paper derives sparse recurrence relations between orthogonal polynomials on a triangle and their partial derivatives, which are analogous to recurrence relations for Jacobi polynomials. We derive these recurrences in a systematic fashion by introducing ladder operators that map an orthogonal polynomial to another by incrementing or decrementing its associated parameters by one.

math.CA↗

On the singular values of matrices with high displacement rank

We introduce a new ADI-based low rank solver for $AX-XB=F$, where $F$ has rapidly decaying singular values. Our approach results in both theoretical and practical gains, including (1) the derivation of new bounds on singular values for classes of matrices with high displacement rank, (2) a practical algorithm for solving certain Lyapunov and Sylvester matrix equations with high rank right-hand sides, and (3) a collection of low rank Poisson solvers that achieve spectral accuracy and optimal computational complexity.

math.NA↗

Fast Poisson solvers for spectral methods

Poisson's equation is the canonical elliptic partial differential equation. While there exist fast Poisson solvers for finite difference and finite element methods, fast Poisson solvers for spectral methods have remained elusive. Here, we derive spectral methods for solving Poisson's equation on a square, cylinder, solid sphere, and cube that have an optimal complexity (up to polylogarithmic terms) in terms of the degrees of freedom required to represent the solution. Whereas FFT-based fast Poisson solvers exploit structured eigenvectors of finite difference matrices, our solver exploits a separated spectra property that holds for our spectral discretizations. Without parallelization, we can solve Poisson's equation on a square with 100 million degrees of freedom in under two minutes on a standard laptop.

math.NA↗

Computing with functions in spherical and polar geometries II. The disk

A collection of algorithms is described for numerically computing with smooth functions defined on the unit disk. Low rank approximations to functions in polar geometries are formed by synthesizing the disk analogue of the double Fourier sphere method with a structure-preserving variant of iterative Gaussian elimination that is shown to converge geometrically for certain analytic functions. This adaptive procedure is near-optimal in its sampling strategy, producing approximants that are stable for differentiation and facilitate the use of FFT-based algorithms in both variables. The low rank form of the approximants is especially useful for operations such as integration and differentiation, reducing them to essentially 1D procedures, and it is also exploited to formulate a new fast disk Poisson solver that computes low rank approximations to solutions. This work complements a companion paper (Part I) on computing with functions on the surface of the unit sphere.

math.NA↗

A nonuniform fast Fourier transform based on low rank approximation

By viewing the nonuniform discrete Fourier transform (NUDFT) as a perturbed version of a uniform discrete Fourier transform, we propose a fast, stable, and simple algorithm for computing the NUDFT that costs $\mathcal{O}(N\log N\log(1/ε)/\log\!\log(1/ε))$ operations based on the fast Fourier transform, where $N$ is the size of the transform and $0<ε<1$ is a working precision. Our key observation is that a NUDFT and DFT matrix divided entry-by-entry is often well-approximated by a low rank matrix, allowing us to express a NUDFT matrix as a sum of diagonally-scaled DFT matrices. Our algorithm is simple to implement, automatically adapts to any working precision, and is competitive with state-of-the-art algorithms. In the fully uniform case, our algorithm is essentially the FFT. We also describe quasi-optimal algorithms for the inverse NUDFT and two-dimensional NUDFTs.

math.NA↗

Vector spaces of linearizations for matrix polynomials: a bivariate polynomial approach

We revisit the landmark paper [D. S. Mackey, N. Mackey, C. Mehl, and V. Mehrmann, SIAM J. Matrix Anal. Appl., 28 (2006), pp.~971--1004] and, by viewing matrices as coefficients for bivariate polynomials, we provide concise proofs for key properties of linearizations for matrix polynomials. We also show that every pencil in the double ansatz space is intrinsically connected to a Bézout matrix, which we use to prove the eigenvalue exclusion theorem. In addition our exposition allows for any polynomial basis and for any field. The new viewpoint also leads to new results. We generalize the double ansatz space by exploiting its algebraic interpretation as a space of Bézout pencils to derive new linearizations with potential applications in the theory of structured matrix polynomials. Moreover, we analyze the conditioning of double ansatz space linearizations in the important practical case of a Chebyshev basis.

math.NA↗

On the singular values of matrices with displacement structure

Matrices with displacement structure such as Pick, Vandermonde, and Hankel matrices appear in a diverse range of applications. In this paper, we use an extremal problem involving rational functions to derive explicit bounds on the singular values of such matrices. For example, we show that the $k$th singular value of a real $n\times n$ positive definite Hankel matrix, $H_n$, is bounded by $Cρ^{-k/\log n}\|H\|_2$ with explicitly given constants $C>0$ and $ρ>1$, where $\|H_n\|_2$ is the spectral norm. This means that a real $n\times n$ positive definite Hankel matrix can be approximated, up to an accuracy of $ε\|H_n\|_2$ with $0<ε<1$, by a rank $\mathcal{O}(\log n\log(1/ε) )$ matrix. Analogous results are obtained for Pick, Cauchy, real Vandermonde, Löwner, and certain Krylov matrices.

math.NA↗

Stable extrapolation of analytic functions

This paper examines the problem of extrapolation of an analytic function for $x > 1$ given perturbed samples from an equally spaced grid on $[-1,1]$. Mathematical folklore states that extrapolation is in general hopelessly ill-conditioned, but we show that a more precise statement carries an interesting nuance. For a function $f$ on $[-1,1]$ that is analytic in a Bernstein ellipse with parameter $ρ> 1$, and for a uniform perturbation level $ε$ on the function samples, we construct an asymptotically best extrapolant $e(x)$ as a least squares polynomial approximant of degree $M^*$ given explicitly. We show that the extrapolant $e(x)$ converges to $f(x)$ pointwise in the interval $I_ρ\in[1,(ρ+ρ^{-1})/2)$ as $ε\to 0$, at a rate given by a $x$-dependent fractional power of $ε$. More precisely, for each $x \in I_ρ$ we have \[ |f(x) - e(x)| = \mathcal{O}\left( ε^{-\log r(x) / \logρ} \right), \qquad\qquad r(x) = \frac{x+\sqrt{x^2-1}}ρ, \] up to log factors, provided that the oversampling conditioning is satisfied. That is, \[ M^* \leq \frac{1}{2} \sqrt{N}, \] which is known to be needed from approximation theory. In short, extrapolation enjoys a weak form of stability, up to a fraction of the characteristic smoothness length. The number of function samples, $N+1$, does not bear on the size of the extrapolation error provided that it obeys the oversampling condition. We also show that one cannot construct an asymptotically more accurate extrapolant from $N+1$ equally spaced samples than $e(x)$, using any other linear or nonlinear procedure. The proofs involve original statements on the stability of polynomial approximation in the Chebyshev basis from equally spaced samples and these are expected to be of independent interest.

cs.IT↗

Fast polynomial transforms based on Toeplitz and Hankel matrices

Many standard conversion matrices between coefficients in classical orthogonal polynomial expansions can be decomposed using diagonally-scaled Hadamard products involving Toeplitz and Hankel matrices. This allows us to derive $\smash{\mathcal{O}(N(\log N)^2)}$ algorithms, based on the fast Fourier transform, for converting coefficients of a degree $N$ polynomial in one polynomial basis to coefficients in another. Numerical results show that this approach is competitive with state-of-the-art techniques, requires no precomputational cost, can be implemented in a handful of lines of code, and is easily adapted to extended precision arithmetic.

math.NA↗

Computing with functions in spherical and polar geometries I. The sphere

A collection of algorithms is described for numerically computing with smooth functions defined on the unit sphere. Functions are approximated to essentially machine precision by using a structure-preserving iterative variant of Gaussian elimination together with the double Fourier sphere method. We show that this procedure allows for stable differentiation, reduces the oversampling of functions near the poles, and converges for certain analytic functions. Operations such as function evaluation, differentiation, and integration are particularly efficient and can be computed by essentially one-dimensional algorithms. A highlight is an optimal complexity direct solver for Poisson's equation on the sphere using a spectral method. Without parallelization, we solve Poisson's equation with $100$ million degrees of freedom in one minute on a standard laptop. Numerical results are presented throughout. In a companion paper (part II) we extend the ideas presented here to computing with functions on the disk.

math.NA↗

Gaussian elimination corrects pivoting mistakes

Gaussian elimination (GE) is the archetypal direct algorithm for solving linear systems of equations and this has been its primary application for thousands of years. In the last decade, GE has found another major use as an iterative algorithm for low rank approximation. In this setting, GE is often employed with complete pivoting and designed to allow for non-optimal pivoting, i.e., pivoting mistakes, that could render GE numerically unstable when implemented in floating point arithmetic. While it may appear that pivoting mistakes could accumulate and lead to a large growth factor, we show that later GE steps correct earlier pivoting mistakes, even while more are being made. In short, GE is very robust to non-optimal pivots, allowing for its iterative variant to flourish.

math.NA↗

Numerical instability of resultant methods for multidimensional rootfinding

Hidden-variable resultant methods are a class of algorithms for solving multidimensional polynomial rootfinding problems. In two dimensions, when significant care is taken, they are competitive practical rootfinders. However, in higher dimensions they are known to miss zeros, calculate roots to low precision, and introduce spurious solutions. We show that the hidden variable resultant method based on the Cayley (Dixon or Bézout) matrix is inherently and spectacularly numerically unstable by a factor that grows exponentially with the dimension. We also show that the Sylvester matrix for solving bivariate polynomial systems can square the condition number of the problem. In other words, two popular hidden variable resultant methods are numerically unstable, and this mathematically explains the difficulties that are frequently reported by practitioners. Regardless of how the constructed polynomial eigenvalue problem is solved, severe numerical difficulties will be present. Along the way, we prove that the Cayley resultant is a generalization of Cramer's rule for solving linear systems and generalize Clenshaw's algorithm to an evaluation scheme for polynomials expressed in a degree-graded polynomial basis.

math.NA↗

A fast FFT-based discrete Legendre transform

An $\mathcal{O}(N(\log N)^2/\log\!\log N)$ algorithm for computing the discrete Legendre transform and its inverse is described. The algorithm combines a recently developed fast transform for converting between Legendre and Chebyshev coefficients with a Taylor series expansion for Chebyshev polynomials about equally-spaced points in the frequency domain. Both components are based on the FFT, and as an intermediate step we obtain an $\mathcal{O}(N\log N)$ algorithm for evaluating a degree $N-1$ Chebyshev expansion at an $N$-point Legendre grid. Numerical results are given to demonstrate performance and accuracy.

math.NA↗