arXiv subjects

Daan Huybrechs

Publications and source records attributed to Daan Huybrechs.

At least 19 recordsLinked to original sources

arXiv2026

How different is rational approximation from piecewise polynomial approximation?

The first aim of this paper is to show that there is merit to the question posed in the title. Indeed, for certain function classes, approximation by rational functions and by piecewise polynomials are surprisingly similar. Between these two, polynomials are more widely studied and more widely used. One setting in which both approaches are equally well understood, and equally practical in use, is that of approximating univariate functions with point singularities. In this context we can fully address the question. We review classical literature on the topic which shows that both approaches do indeed achieve similar convergence rates. However, rational approximations come with significantly smaller constants. Owing to recent advances in practical rational approximation, we can augment the discussion with a comparison of modern numerical techniques that achieve the optimal rates. We end by showing numerically that the difference becomes even more pronounced in several variables.

math.NA↗

arXiv2026

Compact Rational Krylov for Parametrized Systems with Application to BEM Frequency Sweeping

In parametrized linear systems $\mathsf{P}(μ)\mathsf{x}=\mathsf{b}$ the system matrix $\mathsf{P}$ depends nonlinearly on a parameter $μ$ and solutions are sought for many values of this parameter. We show that the compact rational Krylov (CORK) framework, originally introduced to solve nonlinear eigenvalue problems, can be used to efficiently produce approximate solutions to such a system for many values of the parameter at once. In this approach, the parametrized system is first linearized, resulting in a large shifted linear system $(\boldsymbol{\mathsf{A}}-μ\boldsymbol{\mathsf{B}})\boldsymbol{\mathsf{y}}=\boldsymbol{\mathsf{d}}$. We formulate a left- and right-preconditioned rational Krylov GMRES method for shifted linear systems. In the setting of parametrized linear systems, these can exploit the structure in the linearization, and in combination with the CORK framework, computational and memory complexity mainly depend on the problem size, less the degree of the linearization. Additionally, we show how to incorporate a right-hand side $\mathsf{b}(μ)$ that also depends on the parameter, how to choose the shifts to steer convergence and how to allow for inexact solves at these shifts throughout the iterations. As an application we consider the 'frequency sweeping' of Helmholtz scattering problems through the Boundary Element Method (BEM), enabled via an efficient representation of the dense but data-sparse wavenumber-dependent system matrix.

math.NA↗

arXiv2026

Function Approximation in Numerically Rank-Deficient Bases

We study linear function approximation in a finite basis under finite-precision arithmetic. In a highly non-orthogonal basis, certain directions are only weakly represented, so that rounding errors can significantly distort the effectively spanned space. In the first part of the paper, we formalize this phenomenon through the notion of a numerical span. Using a novel model for the rounding errors involved, we prove that approximation in the numerical span behaves like approximation in exact arithmetic subject to an additional penalty proportional to the size of the expansion coefficients and the unit roundoff. A key implication is that straightforward numerical orthogonalization cannot mitigate the effects induced by finite-precision arithmetic. The framework also provides a theoretical justification for $\ell^2$-regularized approximation. Moreover, regularization controls the amplification of rounding errors in the computation of expansion coefficients. In the second part of the paper, we address sampling for function approximation in the presence of numerical rank-deficiency. We demonstrate that regularization has another fundamental benefit: it relaxes the conditions required for accurate least squares approximation from sampled data. This effect is made concrete through an analysis of randomized sampling based on a regularized variant of the Christoffel function. The resulting sample complexity bounds depend on an effective dimension that measures the number of directions that remain useful after finite-precision rounding. We also show that regularization renders the Christoffel function computable in contrast to the standard Christoffel function, whose numerical evaluation may require arbitrarily high precision in the presence of numerical rank-deficiency. We apply the derived theory to obtain new results for the discretization of univariate Fourier extension frames.

math.NA↗

arXiv2025

On the algorithmic construction of deep ReLU networks

It is difficult to describe in mathematical terms what a neural network trained on data represents. On the other hand, there is a growing mathematical understanding of what neural networks are in principle capable of representing. Feedforward neural networks using the ReLU activation function represent continuous and piecewise linear functions and can approximate many others. The study of their expressivity addresses the question: which ones? Contributing to the available answers, we take the perspective of a neural network as an algorithm. In this analogy, a neural network is programmed constructively, rather than trained from data. An interesting example is a sorting algorithm: we explicitly construct a neural network that sorts its inputs exactly, not approximately, and that, in a sense, has optimal computational complexity if the input dimension is large. Such constructed networks may have several billion parameters. We construct and analyze several other examples, both existing and new. We find that, in these examples, neural networks as algorithms are typically recursive and parallel. Compared to conventional algorithms, ReLU networks are restricted by having to be continuous. Moreover, the depth of recursion is limited by the depth of the network, with deep networks having superior properties over shallow ones.

cs.LG↗

arXiv2025

Uniform H-matrix Compression with Applications to Boundary Integral Equations

Boundary integral equations lead to dense system matrices when discretized, yet they are data-sparse. Using the $\mathcal{H}$-matrix format, this sparsity is exploited to achieve $\mathcal{O}(N\log N)$ complexity for storage and multiplication by a vector. This is achieved purely algebraically, based on low-rank approximations of subblocks, and hence the format is also applicable to a wider range of problems. The $\mathcal{H}^2$-matrix format improves the complexity to $\mathcal{O}(N)$ by introducing a recursive structure onto subblocks on multiple levels. However, in many cases this comes with a large proportionality constant, making the $\mathcal{H}^2$-matrix format advantageous mostly for large problems. In this paper we investigate the usefulness of a matrix format that lies in between these two: Uniform $\mathcal{H}$-matrices. An algebraic compression algorithm is introduced to transform a regular $\mathcal{H}$-matrix into a uniform $\mathcal{H}$-matrix, which maintains the asymptotic complexity. Using examples of the BEM formulation of the Helmholtz equation, we show that this scheme lowers the storage requirement and execution time of the matrix-vector product without significantly impacting the construction time.

math.NA↗

arXiv2024

QR-based Parallel Set-Valued Approximation with Rational Functions

In this article a fast and parallelizable algorithm for rational approximation is presented. The method, called (P)QR-AAA, is a (parallel) set-valued variant of the AAA algorithm for scalar functions. It builds on the set-valued AAA framework introduced by Lietaert, Meerbergen, P{é}rez and Vandereycken, accelerating it by using an approximate orthogonal basis obtained from a truncated QR decomposition. We demonstrate both theoretically and numerically this method's accuracy and efficiency. We show how it can be parallelized while maintaining the desired accuracy, with minimal communication cost.

math.NA↗

arXiv2024

Multivariate rational approximation of functions with curves of singularities

Functions with singularities are notoriously difficult to approximate with conventional approximation schemes. In computational applications, they are often resolved with low-order piecewise polynomials, multilevel schemes, or other types of grading strategies. Rational functions are an exception to this rule: for univariate functions with point singularities, such as branch points, rational approximations exist with root-exponential convergence in the rational degree. This is typically enabled by the clustering of poles near the singularity. Both the theory and computational practice of rational functions for function approximation have focused on the univariate case, with extensions to two dimensions via identification with the complex plane. Multivariate rational functions, i.e., quotients of polynomials of several variables, are relatively unexplored in comparison. Yet, apart from a steep increase in theoretical complexity, they also offer a wealth of opportunities. A first observation is that singularities of multivariate rational functions may be continuous curves of poles, rather than isolated ones. By generalizing the clustering of poles from points to curves, we explore constructions of multivariate rational approximations to functions with curves of singularities.

math.NA↗

arXiv2024

Efficient least squares approximation and collocation methods using radial basis functions

We describe an efficient method for the approximation of functions using radial basis functions (RBFs), and extend this to a solver for boundary value problems on irregular domains. The method is based on RBFs with centers on a regular grid defined on a bounding box, with some of the centers outside the computational domain. The equation is discretized using collocation with oversampling, with collocation points inside the domain only, resulting in a rectangular linear system to be solved in a least squares sense. The goal of this paper is the efficient solution of that rectangular system. We show that the least squares problem splits into a regular part, which can be expedited with the FFT, and a low rank perturbation, which is treated separately with a direct solver. The rank of the perturbation is influenced by the irregular shape of the domain and by the weak enforcement of boundary conditions at points along the boundary. The solver extends the AZ algorithm which was previously proposed for function approximation involving frames and other overcomplete sets. The solver has near optimal log-linear complexity for univariate problems, and loses optimality for higher-dimensional problems but remains faster than a direct solver.

math.NA↗

arXiv2023

Resolution of singularities by rational functions

Results on the rational approximation of functions containing singularities are presented. We build further on the ''lightning method'', recently proposed by Trefethen and collaborators, based on exponentially clustering poles close to the singularities. Our results are obtained by augmenting the lightning approximation set with either a low-degree polynomial basis or poles clustering towards infinity, in order to obtain a robust approximation of the smooth behaviour of the function. This leads to a significant increase in the achievable accuracy as well as the convergence rate of the numerical scheme. For the approximation of $x^α$ on $[0,1]$, the optimal convergence rate as shown by Stahl in 1993 is now achieved simply by least-squares fitting.

math.NA↗

arXiv2023

Stable approximation of Helmholtz solutions in the disk by evanescent plane waves

Superpositions of plane waves are known to approximate well the solutions of the Helmholtz equation. Their use in discretizations is typical of Trefftz methods for Helmholtz problems, aiming to achieve high accuracy with a small number of degrees of freedom. However, Trefftz methods lead to ill-conditioned linear systems, and it is often impossible to obtain the desired accuracy in floating-point arithmetic. In this paper we show that a judicious choice of plane waves can ensure high-accuracy solutions in a numerically stable way, in spite of having to solve such ill-conditioned systems. Numerical accuracy of plane wave methods is linked not only to the approximation space, but also to the size of the coefficients in the plane wave expansion. We show that the use of plane waves can lead to exponentially large coefficients, regardless of the orientations and the number of plane waves, and this causes numerical instability. We prove that all Helmholtz fields are continuous superposition of evanescent plane waves, i.e., plane waves with complex propagation vectors associated with exponential decay, and show that this leads to bounded representations. We provide a constructive scheme to select a set of real and complex-valued propagation vectors numerically. This results in an explicit selection of plane waves and an associated Trefftz method that achieves accuracy and stability. The theoretical analysis is provided for a two-dimensional domain with circular shape. However, the principles are general and we conclude the paper with a numerical experiment demonstrating practical applicability also for polygonal domains.

math.NA↗

arXiv2023

Efficient Function Approximation in Enriched Approximation Spaces

An enriched approximation space is the span of a conventional basis with a few extra functions included, for example to capture known features of the solution to a computational problem. Adding functions to a basis makes it overcomplete and, consequently, the corresponding discretized approximation problem may require solving an ill-conditioned system. Recent research indicates that these systems can still provide highly accurate numerical approximations under reasonable conditions. In this paper we propose an efficient algorithm to compute such approximations. It is based on the AZ algorithm for overcomplete sets and frames, which simplifies in the case of an enriched basis. In addition, analysis of the original AZ algorithm and of the proposed variant gives constructive insights on how to achieve optimal and stable discretizations using enriched bases. We apply the algorithm to examples of enriched approximation spaces in literature, including a few non-standard approximation problems and an enriched spectral method for a 2D boundary value problem, and show that the simplified AZ algorithm is indeed stable, accurate and efficient.

math.NA↗

arXiv2023

Sigmoid functions and multiscale resolution of singularities

In this short, conceptual paper we observe that essentially the same mathematics applies in three contexts with disparate literatures: (1) sigmoidal and RBF approximation of smooth functions, (2) rational approximation of analytic functions near singularities, and (3) $hp$ mesh refinement for solution of PDEs. The relationship of (1) and (2) is as simple as the change of variables $s = \log(x)$, and our informal mnemonic for this relationship is ``sigmoid = log(ratapprox).''

math.NA↗

arXiv2022

Stable and accurate least squares radial basis function approximations on bounded domains

The computation of global radial basis function (RBF) approximations requires the solution of a linear system which, depending on the choice of RBF parameters, may be ill-conditioned. We study the stability and accuracy of approximation methods using the Gaussian RBF in all scaling regimes of the associated shape parameter. The approximation is based on discrete least squares with function samples on a bounded domain, using RBF centers both inside and outside the domain. This results in a rectangular linear system. We show for one-dimensional approximations that linear scaling of the shape parameter with the degrees of freedom is optimal, resulting in constant overlap between neighbouring RBF's regardless of their number, and we propose an explicit suitable choice of the proportionality constant. We show numerically that highly accurate approximations to smooth functions can also be obtained on bounded domains in several dimensions, using a linear scaling with the degrees of freedom per dimension. We extend the least squares approach to a collocation-based method for the solution of elliptic boundary value problems and illustrate that the combination of centers outside the domain, oversampling and optimal scaling can result in accuracy close to machine precision in spite of having to solve very ill-conditioned linear systems.

math.NA↗

arXiv2022

On the computation of the SVD of Fourier submatrices

Contiguous submatrices of the Fourier matrix are known to be ill-conditioned. In a recent paper in SIAM Review A. Barnett has provided new bounds on the rate of ill-conditioning of the discrete Fourier submatrices. In this paper we focus on the corresponding singular value decomposition. The singular vectors go by the name of periodic discrete prolate spheroidal sequences (P-DPSS). The singular values exhibit an initial plateau, which depends on the dimensions of the submatrix, after which they decay rapidly. The latter regime is known as the plunge region and it is compatible with the submatrices being ill-conditioned. The discrete prolate sequences have received much less study than their continuous counterparts, prolate spheroidal wave functions, associated with continuous Fourier transforms and widely studied following the work of Slepian in the 1970's. In this paper we collect and expand known results on the stable numerical computation of the singular values and vectors of Fourier submatrices. We illustrate the computations and point out a few applications in which Fourier submatrices arise.

math.NA↗

arXiv2022

Arbitrary-order asymptotic expansions of Gaussian quadrature rules with classical and generalised weight functions

Gaussian quadrature rules are a classical tool for the numerical approximation of integrals with smooth integrands and positive weight functions. We derive and expicitly list asymptotic expressions for the points and weights of Gaussian quadrature rules for three general classes of positive weight functions: analytic functions on a bounded interval with algebraic singularities at the endpoints, analytic weight functions on the halfline with exponential decay at infinity and an algebraic singularity at the finite endpoint, and analytic functions on the real line with exponential decay in both directions at infinity. The results include the Gaussian rules of classical orthogonal polynomials (Legendre, Jacobi, Laguerre and Hermite) as special cases. We present experiments indicating the range of the number of points at which these expressions achieve high precision. We provide an algorithm that can compute arbitrarily many terms in these expansions for the classical cases, and many though not all terms for the generalized cases.

math.NA↗

arXiv2022

AAA interpolation of equispaced data

We propose AAA rational approximation as a method for interpolating or approximating smooth functions from equispaced data samples. Although it is always better to approximate from large numbers of samples if they are available, whether equispaced or not, this method often performs impressively even when the sampling grid is fairly coarse. In most cases it gives more accurate approximations than other methods.

math.NA↗

arXiv2022

Frequency extraction for BEM-matrices arising from the 3D scalar Helmholtz equation

The discretisation of boundary integral equations for the scalar Helmholtz equation leads to large dense linear systems. Efficient boundary element methods (BEM), such as the fast multipole method (FMM) and $\Hmat$ based methods, focus on structured low-rank approximations of subblocks in these systems. It is known that the ranks of these subblocks increase linearly with the wavenumber. We explore a data-sparse representation of BEM-matrices valid for a range of frequencies, based on extracting the known phase of the Green's function. Algebraically, this leads to a Hadamard product of a frequency matrix with an $\Hmat$. We show that the frequency dependency of this $\Hmat$ can be determined using a small number of frequency samples, even for geometrically complex three-dimensional scattering obstacles. We describe an efficient construction of the representation by combining adaptive cross approximation with adaptive rational approximation in the continuous frequency dimension. We show that our data-sparse representation allows to efficiently sample the full BEM-matrix at any given frequency, and as such it may be useful as part of an efficient sweeping routine.

math.NA↗

arXiv2022

An analysis of least-squares oversampled collocation methods for compactly perturbed boundary integral equations in two dimensions

In recent work (Maierhofer & Huybrechs, 2022, Adv. Comput. Math.), the authors showed that least-squares oversampling can improve the convergence properties of collocation methods for boundary integral equations involving operators of certain pseudo-differential form. The underlying principle is that the discrete method approximates a Bubnov$-$Galerkin method in a suitable sense. In the present work, we extend this analysis to the case when the integral operator is perturbed by a compact operator $\mathcal{K}$ which is continuous as a map on Sobolev spaces on the boundary, $\mathcal{K}:H^{p}\rightarrow H^{q}$ for all $p,q\in\mathbb{R}$. This study is complicated by the fact that both the test and trial functions in the discrete Bubnov-Galerkin orthogonality conditions are modified over the unperturbed setting. Our analysis guarantees that previous results concerning optimal convergence rates and sufficient rates of oversampling are preserved in the more general case. Indeed, for the first time, this analysis provides a complete explanation of the advantages of least-squares oversampled collocation for boundary integral formulations of the Laplace equation on arbitrary smooth Jordan curves in 2D. Our theoretical results are shown to be in very good agreement with numerical experiments.

math.NA↗