arXiv subjects

Alex Townsend

Publications and source records attributed to Alex Townsend.

At least 55 records · Page 3Linked to original sources

arXiv2022

Learning elliptic partial differential equations with randomized linear algebra

Given input-output pairs of an elliptic partial differential equation (PDE) in three dimensions, we derive the first theoretically-rigorous scheme for learning the associated Green's function $G$. By exploiting the hierarchical low-rank structure of $G$, we show that one can construct an approximant to $G$ that converges almost surely and achieves a relative error of $\mathcal{O}(Γ_ε^{-1/2}\log^3(1/ε)ε)$ using at most $\mathcal{O}(ε^{-6}\log^4(1/ε))$ input-output training pairs with high probability, for any $0<ε<1$. The quantity $0<Γ_ε\leq 1$ characterizes the quality of the training dataset. Along the way, we extend the randomized singular value decomposition algorithm for learning matrices to Hilbert--Schmidt operators and characterize the quality of covariance kernels for PDE learning.

math.NA↗

arXiv2021

Data-driven Algorithms for signal processing with trigonometric rational functions

Rational approximation schemes for reconstructing periodic signals from samples with poorly separated spectral content are described. These methods are automatic and adaptive, requiring no tuning or manual parameter selection. Collectively, they form a framework for fitting trigonometric rational models to data that is robust to various forms of corruption, including additive Gaussian noise, perturbed sampling grids, and missing data. Our approach combines a variant of Prony's method with a modified version of the AAA algorithm. Using representations in both frequency and time space, a collection of algorithms is described for adaptively computing with trigonometric rationals. This includes procedures for differentiation, filtering, convolution, and more. A new MATLAB software system based on these algorithms is introduced. Its effectiveness is illustrated with synthetic and practical examples drawn from applications including biomedical monitoring, acoustic denoising, and feature detection.

math.NA↗

arXiv2021

Parallel algorithms for computing the tensor-train decomposition

The tensor-train (TT) decomposition expresses a tensor in a data-sparse format used in molecular simulations, high-order correlation functions, and optimization. In this paper, we propose four parallelizable algorithms that compute the TT format from various tensor inputs: (1) Parallel-TTSVD for traditional format, (2) PSTT and its variants for streaming data, (3) Tucker2TT for Tucker format, and (4) TT-fADI for solutions of Sylvester tensor equations. We provide theoretical guarantees of accuracy, parallelization methods, scaling analysis, and numerical results. For example, for a $d$-dimension tensor in $\mathbb{R}^{n\times\dots\times n}$, a two-sided sketching algorithm PSTT2 is shown to have a memory complexity of $\mathcal{O}(n^{\lfloor d/2 \rfloor})$, improving upon $\mathcal{O}(n^{d-1})$ from previous algorithms.

math.NA↗

arXiv2021

Arbitrary-Depth Universal Approximation Theorems for Operator Neural Networks

The standard Universal Approximation Theorem for operator neural networks (NNs) holds for arbitrary width and bounded depth. Here, we prove that operator NNs of bounded width and arbitrary depth are universal approximators for continuous nonlinear operators. In our main result, we prove that for non-polynomial activation functions that are continuously differentiable at a point with a nonzero derivative, one can construct an operator NN of width five, whose inputs are real numbers with finite decimal representations, that is arbitrarily close to any given continuous nonlinear operator. We derive an analogous result for non-affine polynomial activation functions. We also show that depth has theoretical advantages by constructing operator ReLU NNs of depth $2k^3+8$ and constant width that cannot be well-approximated by any operator ReLU NN of depth $k$, unless its width is exponential in $k$.

cs.LG↗

arXiv2021

Anderson Acceleration Based on the $\mathcal{H}^{-s}$ Sobolev Norm for Contractive and Noncontractive Fixed-Point Operators

Anderson acceleration (AA) is a technique for accelerating the convergence of fixed-point iterations. In this paper, we apply AA to a sequence of functions and modify the norm in its internal optimization problem to the $\mathcal{H}^{-s}$ norm, for some positive integer $s$, to bias it towards low-frequency spectral content in the residual. We analyze the convergence of AA by quantifying its improvement over Picard iteration. We find that AA based on the $\mathcal{H}^{-2}$ norm is well-suited to solve fixed-point operators derived from second-order elliptic differential operators, including the Helmholtz equation.

math.NA↗

arXiv2021

Sufficiently dense Kuramoto networks are globally synchronizing

Consider any network of $n$ identical Kuramoto oscillators in which each oscillator is coupled bidirectionally with unit strength to at least $μ(n-1)$ other oscillators. There is a critical value of the connectivity, $μ_c$, such that whenever $μ>μ_c$, the system is guaranteed to converge to the all-in-phase synchronous state for almost all initial conditions, but when $μ<μ_c$, there are networks with other stable states. The precise value of the critical connectivity remains unknown, but it has been conjectured to be $μ_c=0.75$. In 2020, Lu and Steinerberger proved that $μ_c\leq 0.7889$, and Yoneda, Tatsukawa, and Teramae proved in 2021 that $μ_c > 0.6838$. In this paper, we prove that $μ_c\leq 0.75$ and explain why this is the best upper bound that one can obtain by a purely linear stability analysis.

math.DS↗

arXiv2021

An optimal complexity spectral method for Navier--Stokes simulations in the ball

We develop a spectral method for solving the incompressible generalized Navier--Stokes equations in the ball with no-flux and prescribed slip boundary conditions. The algorithm achieves an optimal complexity per time step of $\mathcal{O}(N\log^2(N))$, where $N$ is the number of spatial degrees of freedom. The method relies on the poloidal-toroidal decomposition of solenoidal vector fields, the double Fourier sphere method, the Fourier and ultraspherical spectral method, and the spherical harmonics transform to decouple the Navier--Stokes equations and achieve the desired complexity and spectral accuracy.

math.NA↗

arXiv2020

The ultraspherical spectral element method

We introduce a novel spectral element method based on the ultraspherical spectral method and the hierarchical Poincaré-Steklov scheme for solving second-order linear partial differential equations on polygonal domains with unstructured quadrilateral or triangular meshes. Properties of the ultraspherical spectral method lead to almost banded linear systems, allowing the element method to be competitive in the high-polynomial regime ($p > 5$). The hierarchical Poincaré-Steklov scheme enables precomputed solution operators to be reused, allowing for fast elliptic solves in implicit and semi-implicit time-steppers. The resulting spectral element method achieves an overall computational complexity of $\mathcal{O}(p^4/h^3)$ for mesh size $h$ and polynomial order $p$, enabling $hp$-adaptivity to be efficiently performed. We develop an open-source software system, ultraSEM, for flexible, user-friendly spectral element computations in MATLAB.

math.NA↗

arXiv2020

Computing spectral measures of self-adjoint operators

Using the resolvent operator, we develop an algorithm for computing smoothed approximations of spectral measures associated with self-adjoint operators. The algorithm can achieve arbitrarily high-orders of convergence in terms of a smoothing parameter for computing spectral measures of general differential, integral, and lattice operators. Explicit pointwise and $L^p$-error bounds are derived in terms of the local regularity of the measure. We provide numerical examples, including a partial differential operator, a magnetic tight-binding model of graphene, and compute one thousand eigenvalues of a Dirac operator to near machine precision without spectral pollution. The algorithm is publicly available in $\texttt{SpecSolve}$, which is a software package written in MATLAB.

math.NA↗

arXiv2020

Over-parametrized neural networks as under-determined linear systems

We draw connections between simple neural networks and under-determined linear systems to comprehensively explore several interesting theoretical questions in the study of neural networks. First, we emphatically show that it is unsurprising such networks can achieve zero training loss. More specifically, we provide lower bounds on the width of a single hidden layer neural network such that only training the last linear layer suffices to reach zero training loss. Our lower bounds grow more slowly with data set size than existing work that trains the hidden layer weights. Second, we show that kernels typically associated with the ReLU activation function have fundamental flaws -- there are simple data sets where it is impossible for widely studied bias-free models to achieve zero training loss irrespective of how the parameters are chosen or trained. Lastly, our analysis of gradient descent clearly illustrates how spectral properties of certain matrices impact both the early iteration and long-term training behavior. We propose new activation functions that avoid the pitfalls of ReLU in that they admit zero training loss solutions for any set of distinct data points and experimentally exhibit favorable spectral properties.

math.NA↗

arXiv2020

Rational neural networks

We consider neural networks with rational activation functions. The choice of the nonlinear activation function in deep learning architectures is crucial and heavily impacts the performance of a neural network. We establish optimal bounds in terms of network complexity and prove that rational neural networks approximate smooth functions more efficiently than ReLU networks with exponentially smaller depth. The flexibility and smoothness of rational activation functions make them an attractive alternative to ReLU, as we demonstrate with numerical experiments.

cs.NE↗

arXiv2020

Suppression of Electroconvective and Morphological Instabilities by an Imposed Cross Flow of the Electrolyte

Electroconvection and its coupling with a morphological instability are important in many applications, including electrodialysis, batteries and fuel cells. In this work, we study the effects of a two-dimensional channel flow on the electroconvective and morphological instabilities using two approaches. In the bulk analysis, we consider the instability of the electroneutral bulk region driven by a second kind electroosmosis slip velocity boundary condition and derive the asymptotic solutions for small and large wavenumbers. In the full analysis, we consider the entire region of the liquid electrolyte and use the ultraspherical spectral method to numerically solve the eigenvalue problems. Both studies show that the imposed flow significantly affects the electroconvective instability. The imposed flow generates a shielding effect by deforming the perturbed ion concentration field and hinders the ion transfer from low- to high- concentration regions which causes the instability. It fully suppresses the electroconvective instability at small wavenumbers and reduces the growth rate of the perturbations at large wavenumbers. The direct effect of the flow on the morphological instability is minor, while the suppression of the electroconvective instability may change the wavenumber of the most unstable mode of the coupled instabilities. For the electroconvective instability, the bulk analysis is qualitatively different from the full analysis at high wavenumbers. For the morphological instability, good agreement is found between the two studies at both small and large wavenumbers.

physics.flu-dyn↗

arXiv2020

Error localization of best L1 polynomial approximants

An important observation in compressed sensing is that the $\ell_0$ minimizer of an underdetermined linear system is equal to the $\ell_1$ minimizer when there exists a sparse solution vector and a certain restricted isometry property holds. Here, we develop a continuous analogue of this observation and show that the best $L_0$ and $L_1$ polynomial approximants of a polynomial that is corrupted on a set of small measure are nearly equal. We go on to demonstrate an error localization property of best $L_1$ polynomial approximants and use our observations to develop an improved algorithm for computing best $L_1$ polynomial approximants to continuous functions.

math.NA↗

arXiv2020

Dense networks that do not synchronize and sparse ones that do

For any network of identical Kuramoto oscillators with identical positive coupling, there is a critical connectivity above which the system is guaranteed to converge to the in-phase synchronous state, for almost all initial conditions. But the precise value of this critical connectivity remains unknown. In 2018, Ling, Xu, and Bandeira proved that if each oscillator is coupled to at least 79.29 percent of all the others, global synchrony is ensured. In 2019, Lu and Steinerberger improved this bound to 78.89 percent. Here, by focusing on circulant networks, we find clues that the critical connectivity may be exactly 75 percent. Our methods yield a slight improvement on the best known lower bound on the critical connectivity, from $68.18\%$ to $68.28\%$. We also consider the opposite end of the connectivity spectrum, where the networks are sparse rather than dense. In this regime, we ask how few edges one needs to add to a ring of $n$ oscillators to turn it into a globally synchronizing network. We prove a partial result: all the twisted states in a ring of size $n=2^m$ can be destabilized by adding just $\mathcal{O}(n \log_2 n)$ edges. To finish the proof, one also needs to rule out all other candidate attractors. We have done this for $n=8$ with computational algebraic geometry, but the problem remains open for larger $n$. Thus, even for systems as simple as Kuramoto oscillators, much remains to be learned about dense networks that do not globally synchronize and sparse ones that do.

nlin.AO↗

arXiv2020

Bounding Zolotarev numbers using Faber rational functions

By closely following a construction by Ganelius, we construct Faber rational functions that allow us to derive tight and explicit bounds on Zolotarev numbers. We use our results to bound the singular values of matrices, including complex-valued Cauchy matrices and Vandermonde matrices with nodes inside the unit disk. We construct Faber rational functions using doubly-connected conformal maps and use their zeros and poles to supply shift parameters in the alternating direction implicit method.

math.CV↗

arXiv2020

FEAST for differential eigenvalue problems

An operator analogue of the FEAST matrix eigensolver is developed to compute the discrete part of the spectrum of a differential operator in a region of interest in the complex plane. Unbounded search regions are handled with a novel rational filter for the right half-plane. If the differential operator is normal or self-adjoint, then the operator analogue preserves that structure and robustly computes eigenvalues to near machine precision accuracy. The algorithm is particularly adept at computing high-frequency modes of differential operators that possess self-adjoint structure with respect to weighted Hilbert spaces.

math.NA↗

arXiv2020

On the compressibility of tensors

Tensors are often compressed by expressing them in low rank tensor formats. In this paper, we develop three methodologies that bound the compressibility of a tensor: (1) Algebraic structure, (2) Smoothness, and (3) Displacement structure. For each methodology, we derive bounds on storage costs that partially explain the abundance of compressible tensors in applied mathematics. For example, we show that the solution tensor $\mathcal{X} \in \mathbb{C}^{n \times n \times n}$ of a discretized Poisson equation $-\nabla^2 u =1$ on $[-1,1]^3$ with zero Dirichlet conditions can be approximated to a relative accuracy of $0<ε<1$ in the Frobenius norm by a tensor in tensor-train format with $\mathcal{O}(n (\log n)^2 (\log(1/ε))^2)$ degrees of freedom. As this bound is constructive, we are also able to solve this equation spectrally with $\mathcal{O}(n (\log n)^3 (\log(1/ε))^3)$ complexity.

math.NA↗

arXiv2019

Computing with functions in the ball

A collection of algorithms in object-oriented MATLAB is described for numerically computing with smooth functions defined on the unit ball in the Chebfun software. Functions are numerically and adaptively resolved to essentially machine precision by using a three-dimensional analogue of the double Fourier sphere method to form "ballfun" objects. Operations such as function evaluation, differentiation, integration, fast rotation by an Euler angle, and a Helmholtz solver are designed. Our algorithms are particularly efficient for vector calculus operations, and we describe how to compute the poloidal-toroidal and Helmholtz--Hodge decomposition of a vector field defined on the ball.

math.NA↗