arXiv subjects

Jamie M. Taylor

Publications and source records attributed to Jamie M. Taylor.

At least 19 recordsLinked to original sources

arXiv2026

Robust Deep FOSLS for Transmission Problems

This work presents a robust, energy-based deep learning framework for solving transmission problems in heterogeneous media, including cases with discontinuous material scenarios. We introduce a weighted First-Order System Least-Squares (FOSLS) formulation involving an energy-norm Poincaré constant and prove its equivalence to a natural energy norm of the underlying equations, with constants independent of material parameters. As a result, the optimization landscape remains aligned with a meaningful error approximation even under high material contrast, where standard neural network losses often deteriorate. We further prove that the FOSLS formulation, together with its integral-loss representation, exhibits a passive variance reduction property, whereby the gradient variance progressively decreases as the loss diminishes, in contrast to methods such as VPINNs and Deep Ritz. From a numerical standpoint, we adopt a reduced-order perspective by constructing a low-dimensional space described by a neural network. The optimal coefficients are computed via a least-squares solver, and the space is subsequently improved through gradient-based updates. By selecting the activation function ReQU, the method mitigates the spurious overshoots typically observed in smooth networks when approximating discontinuities. Numerical experiments in 1D and 2D interface settings corroborate these findings.

math.NA↗

arXiv2026

RUNNs: Ritz-Uzawa Neural Networks for Solving Variational Problems

Solving Partial Differential Equations (PDEs) using neural networks presents different challenges, including integration errors and spectral bias, often leading to poor approximations. In addition, standard neural network-based methods, such as Physics-Informed Neural Networks (PINNs), often lack stability when dealing with PDEs characterized by low-regularity solutions. To address these limitations, we introduce the Ritz--Uzawa Neural Networks (RUNNs) framework, an iterative methodology to solve strong, weak, and ultra-weak variational formulations. Rewriting the PDE as a sequence of Ritz-type minimization problems within a Uzawa loop provides an iterative framework that, in specific cases, reduces both bias and variance during training. We demonstrate that the strong formulation offers a passive variance reduction mechanism, whereas variance remains persistent in weak and ultra-weak regimes. Furthermore, we address the spectral bias of standard architectures through a data-driven frequency tuning strategy. By initializing a Sinusoidal Fourier Feature Mapping based on the Normalized Cumulative Power Spectral Density (NCPSD) of previous residuals or their proxies, the network dynamically adapts its bandwidth to capture high-frequency components and severe singularities. Numerical experiments demonstrate the robustness of RUNNs, accurately resolving highly oscillatory solutions and successfully recovering a discontinuous $L^2$ solution from a distributional $H^{-2}$ source -- a scenario where standard energy-based methods fail.

math.NA↗

arXiv2025

Efficient Numerical Integration for Finite Element Trunk Spaces in 2D and 3D using Machine Learning: A new Optimisation Paradigm to Construct Application-Specific Quadrature Rules

Finite element methods usually construct basis functions and quadrature rules for multidimensional domains via tensor products of one-dimensional counterparts. While straightforward, this approach results in integration spaces larger than necessary, especially as the polynomial degree $p$ or the spatial dimension increases, leading to considerable computational overhead. This work starts from the hypothesis that reducing the dimensionality of the polynomial space can lead to quadrature rules with fewer points and lower computational cost, while preserving the exactness of numerical integration. We use trunk spaces that exclude high-degree monomials that do not improve the approximation quality of the discrete space. These reduced spaces retain sufficient expressive power and allow us to construct smaller (more economical) integration domains. Given a maximum degree $p$, we define trial and test spaces $U$ and $V$ as 2D or 3D trunk spaces and form the integration space $\mathcal{S} = U \otimes V$. We then construct exact quadrature rules by solving a non-convex optimisation problem over the number of points $q$, their coordinates, and weights. We use a shallow neural network with linear activations to parametrise the rule, and a random restart strategy to mitigate convergence to poor local minima. When necessary, we dynamically increase $q$ to achieve exact integration. Our construction reaches machine-precision accuracy (errors below 1e-22) using significantly fewer points than standard tensor-product Gaussian quadrature: up to 30\% reduction in 2D for $p \leq 10$, and 50\% in 3D for $p \leq 6$. These results show that combining the mathematical understanding of polynomial structure with numerical optimisation can lead to a practical and extensible methodology for improving the adaptiveness, efficiency, and scalability of quadrature rules for high-order finite element simulations.

math.NA↗

arXiv2025

Stochastic Quadrature Rules for Solving PDEs using Neural Networks

We examine the challenges associated with numerical integration when applying Neural Networks to solve Partial Differential Equations (PDEs). We specifically investigate the Deep Ritz Method (DRM), chosen for its practical applicability and known sensitivity to integration inaccuracies. Our research demonstrates that both standard deterministic integration techniques and biased stochastic quadrature methods can lead to incorrect solutions. In contrast, employing high-order, unbiased stochastic quadrature rules defined on integration meshes in low dimensions is shown to significantly enhance convergence rates at a comparable computational expense with respect to low-order methods like Monte Carlo. Additionally, we introduce novel stochastic quadrature approaches designed for triangular and tetrahedral mesh elements, offering increased adaptability for handling complex geometric domains. We highlight that the variance inherent in the stochastic gradient acts as a bottleneck for convergence. Furthermore, we observe that for gradient-based optimisation, the crucial factor is the accurate integration of the gradient, rather than just minimizing the quadrature error of the loss function itself.

math.NA↗

arXiv2025

Optimizing Variational Physics-Informed Neural Networks Using Least Squares

Variational Physics-Informed Neural Networks often suffer from poor convergence when using stochastic gradient-descent-based optimizers. By introducing a Least Squares solver for the weights of the last layer of the neural network, we improve the convergence of the loss during training in most practical scenarios. This work analyzes the computational cost of the resulting hybrid Least-Squares/Gradient-Descent optimizer and explains how to implement it efficiently. In particular, we show that a traditional implementation based on backward-mode automatic differentiation leads to a prohibitively expensive algorithm. To remedy this, we propose using either forward-mode automatic differentiation or an ultraweak-type scheme that avoids the differentiation of trial functions in the discrete weak formulation. The proposed alternatives are up to one hundred times faster than the traditional one, recovering a computational cost-per-iteration similar to that of a conventional gradient-descent-based optimizer alone. To support our analysis, we derive computational estimates and conduct numerical experiments in one- and two-dimensional problems.

math.NA↗

arXiv2025

A Least-Squares-Based Neural Network (LS-Net) for Solving Linear Parametric PDEs

Developing efficient methods for solving parametric partial differential equations is crucial for addressing inverse problems. This work introduces a Least-Squares-based Neural Network (LS-Net) method for solving linear parametric PDEs. It utilizes a separated representation form for the parametric PDE solution via a deep neural network and a least-squares solver. In this approach, the output of the deep neural network consists of a vector-valued function, interpreted as basis functions for the parametric solution space, and the least-squares solver determines the optimal solution within the constructed solution space for each given parameter. The LS-Net method requires a quadratic loss function for the least-squares solver to find optimal solutions given the set of basis functions. In this study, we consider loss functions derived from the Deep Fourier Residual and Physics-Informed Neural Networks approaches. We also provide theoretical results similar to the Universal Approximation Theorem, stating that there exists a sufficiently large neural network that can theoretically approximate solutions of parametric PDEs with the desired accuracy. We illustrate the LS-net method by solving one- and two-dimensional problems. Numerical results clearly demonstrate the method's ability to approximate parametric solutions.

math.NA↗

arXiv2024

Regularity-Conforming Neural Networks (ReCoNNs) for solving Partial Differential Equations

Whilst the Universal Approximation Theorem guarantees the existence of approximations to Sobolev functions -- the natural function spaces for PDEs -- by Neural Networks (NNs) of sufficient size, low-regularity solutions may lead to poor approximations in practice. For example, classical fully-connected feed-forward NNs fail to approximate continuous functions whose gradient is discontinuous when employing strong formulations like in Physics Informed Neural Networks (PINNs). In this article, we propose the use of regularity-conforming neural networks, where a priori information on the regularity of solutions to PDEs can be employed to construct proper architectures. We illustrate the potential of such architectures via a two-dimensional (2D) transmission problem, where the solution may admit discontinuities in the gradient across interfaces, as well as power-like singularities at certain points. In particular, we formulate the weak transmission problem in a PINNs-like strong formulation with interface and continuity conditions. Such architectures are partially explainable; discontinuities are explicitly described, allowing the introduction of novel terms into the loss function. We demonstrate via several model problems in one and two dimensions the advantages of using regularity-conforming architectures in contrast to classical architectures. The ideas presented in this article easily extend to problems in higher dimensions.

math.NA↗

arXiv2024

Adaptive Deep Fourier Residual method via overlapping domain decomposition

The Deep Fourier Residual (DFR) method is a specific type of variational physics-informed neural networks (VPINNs). It provides a robust neural network-based solution to partial differential equations (PDEs). The DFR strategy is based on approximating the dual norm of the weak residual of a PDE. This is equivalent to minimizing the energy norm of the error. To compute the dual of the weak residual norm, the DFR method employs an orthonormal spectral basis of the test space, which is known for rectangles or cuboids for multiple function spaces. In this work, we extend the DFR method with ideas of traditional domain decomposition (DD). This enables two improvements: (a) to solve problems in more general polygonal domains, and (b) to develop an adaptive refinement technique in the test space using a Dofler marking algorithm. In the former case, we show that under non-restrictive assumptions we retain the desirable equivalence between the employed loss function and the H1-error, numerically demonstrating adherence to explicit bounds in the case of the L-shaped domain problem. In the latter, we show how refinement strategies lead to potentially significant improvements against a reference, classical DFR implementation with a test function space of significantly lower dimensionality, allowing us to better approximate singular solutions at a more reasonable computational cost.

math.NA↗

arXiv2023

A confined rod: mean field theory for hard rod-like particles

In this paper, we model the configurations of a system of hard rods by viewing each rod in a cell formed by its neighbors. By minimizing the free energy in the model and performing molecular dynamics, where, in both cases, the shape of the cell is a free parameter, we obtain the equilibrium orientational order parameter, free energy and pressure of the system. Our model enables the calculation of anisotropic stresses exerted on the walls of the cell due to shape change of the rod in photoisomerization. These results are a key step towards understanding molecular shape change effects in photomechanical systems under illumination.

cond-mat.soft↗

arXiv2023

Deep Fourier Residual method for solving time-harmonic Maxwell's equations

Solving PDEs with machine learning techniques has become a popular alternative to conventional methods. In this context, Neural networks (NNs) are among the most commonly used machine learning tools, and in those models, the choice of an appropriate loss function is critical. In general, the main goal is to guarantee that minimizing the loss during training translates to minimizing the error in the solution at the same rate. In this work, we focus on the time-harmonic Maxwell's equations, whose weak formulation takes H(curl) as the space of test functions. We propose a NN in which the loss function is a computable approximation of the dual norm of the weak-form PDE residual. To that end, we employ the Helmholtz decomposition of the space H(curl) and construct an orthonormal basis for this space in two and three spatial dimensions. Here, we use the Discrete Sine/Cosine Transform to accurately and efficiently compute the discrete version of our proposed loss function. Moreover, in the numerical examples we show a high correlation between the proposed loss function and the H(curl)-norm of the error, even in problems with low-regularity solutions.

math.NA↗

arXiv2023

Machine Learning Discovery of Optimal Quadrature Rules for Isogeometric Analysis

We propose the use of machine learning techniques to find optimal quadrature rules for the construction of stiffness and mass matrices in isogeometric analysis (IGA). We initially consider 1D spline spaces of arbitrary degree spanned over uniform and non-uniform knot sequences, and then the generated optimal rules are used for integration over higher-dimensional spaces using tensor product sense. The quadrature rule search is posed as an optimization problem and solved by a machine learning strategy based on gradient-descent. However, since the optimization space is highly non-convex, the success of the search strongly depends on the number of quadrature points and the parameter initialization. Thus, we use a dynamic programming strategy that initializes the parameters from the optimal solution over the spline space with a lower number of knots. With this method, we found optimal quadrature rules for spline spaces when using IGA discretizations with up to 50 uniform elements and polynomial degrees up to 8, showing the generality of the approach in this scenario. For non-uniform partitions, the method also finds an optimal rule in a reasonable number of test cases. We also assess the generated optimal rules in two practical case studies, namely, the eigenvalue problem of the Laplace operator and the eigenfrequency analysis of freeform curved beams, where the latter problem shows the applicability of the method to curved geometries. In particular, the proposed method results in savings with respect to traditional Gaussian integration of up to 44% in 1D, 68% in 2D, and 82% in 3D spaces.

math.NA↗

arXiv2022

A Deep Fourier Residual Method for solving PDEs using Neural Networks

When using Neural Networks as trial functions to numerically solve PDEs, a key choice to be made is the loss function to be minimised, which should ideally correspond to a norm of the error. In multiple problems, this error norm coincides with--or is equivalent to--the $H^{-1}$-norm of the residual; however, it is often difficult to accurately compute it. This work assumes rectangular domains and proposes the use of a Discrete Sine/Cosine Transform to accurately and efficiently compute the $H^{-1}$ norm. The resulting Deep Fourier-based Residual (DFR) method efficiently and accurately approximate solutions to PDEs. This is particularly useful when solutions lack $H^{2}$ regularity and methods involving strong formulations of the PDE fail. We observe that the $H^1$-error is highly correlated with the discretised loss during training, which permits accurate error estimation via the loss.

math.NA↗

arXiv2021

On quadrature rules for solving Partial Differential Equations using Neural Networks

Neural Networks have been widely used to solve Partial Differential Equations. These methods require to approximate definite integrals using quadrature rules. Here, we illustrate via 1D numerical examples the quadrature problems that may arise in these applications and propose different alternatives to overcome them, namely: Monte Carlo methods, adaptive integration, polynomial approximations of the Neural Network output, and the inclusion of regularization terms in the loss. We also discuss the advantages and limitations of each proposed alternative. We advocate the use of Monte Carlo methods for high dimensions (above 3 or 4), and adaptive integration or polynomial approximations for low dimensions (3 or below). The use of regularization terms is a mathematically elegant alternative that is valid for any spacial dimension, however, it requires certain regularity assumptions on the solution and complex mathematical analysis when dealing with sophisticated Neural Networks.

math.NA↗

arXiv2021

Effective surface energies in nematic liquid crystals as homogenised rugosity effects

We study the effect of boundary rugosity in nematic liquid crystalline systems. We consider a highly general formulation of the problem, able to simultaneously deal with several liquid crystal theories. We use techniques of Gamma convergence and demonstrate that the effect of fine-scale surface oscillations may be replaced by an effective homogenised surface energy on a simpler domain. The homogenisation limit is then quantitatively studied in a simplified setting, obtaining convergence rates.

math.AP↗

arXiv2021

Hölder regularity and convergence for a non-local model of nematic liquid crystals in the large-domain limit

We consider a non-local free energy functional, modelling a competition between entropy and pairwise interactions reminiscent of the second order virial expansion, with applications to nematic liquid crystals as a particular case. We build on previous work on understanding the behaviour of such models within the large-domain limit, where minimisers converge to minimisers of a quadratic elastic energy with manifold-valued constraint, analogous to harmonic maps. We extend this work to establish Hölder bounds for (almost-)minimisers on bounded domains, and demonstrate stronger convergence of (almost)-minimisers away from the singular set of the limit solution. The proof techniques bear analogy with recent work of singularly perturbed energy functionals, in particular in the context of the Ginzburg-Landau and Landau-de Gennes models.

math.AP↗

arXiv2021

Cavity volume and free energy in many-body systems

Within this work we derive and analyse an expression for the free energy of a single-species system in the thermodynamic limit in terms of a generalised cavity volume, that is exact in general, and in principle applicable to systems across their entire range of density, as well as to particles within a general coordinate space. This provides a universal equation of state, and can thus relate the cavity volume to classical results, such as Mayer's cluster expansions. Through this we are able to provide some insight into the connections between cavity volume and free energy density, as well as their consequences. We use examples which permit explicit computations to further probe these results, reclaiming the exact results for a classical Tonks gas and providing a novel derivation of Onsager's free energy for a single species, isotropic system. Given the complexity of the problem we also provide a local lattice ansatz, exact in one dimension, with which we may approximate the cavity volume for hard sphere systems to provide an accurate equation of state in the cases of hard disks and spheres in both dilute regimes as well as beyond the freezing transition.

cond-mat.stat-mech↗

arXiv2021

Leaky Cell Model of Hard Spheres

We study packings of hard spheres on lattices. The partition function, and therefore the pressure, may be written solely in terms of the accessible free volume, i.e. the volume of space that a sphere can explore without touching another sphere. We compute these free volumes using a leaky cell model, in which the accessible space accounts for the possibility that spheres may escape from the local cage of lattice neighbors. We describe how elementary geometry may be used to calculate the free volume exactly for this leaky cell model in two- and three-dimensional lattice packings and compare the results to the well-known Carnahan-Starling and Percus-Yevick liquid models. We provide formulas for the free volumes of various lattices and use the common tangent construction to identify several phase transitions between them in the leaky cell regime, indicating the possibility of coexistence in crystalline materials.

cond-mat.soft↗

arXiv2019

$Γ$-convergence of a mean-field model of a chiral doped nematic liquid crystal to the Oseen-Frank description of cholesterics

Systems of elongated molecules, doped with small amounts of molecules lacking mirror symmetry can form macroscopically twisted cholesteric liquid crystal phases. The aim of this work is to rigorously derive the Oseen-Frank model of cholesterics from a more fundamental model concerned with pairwise molecular interactions. A non-local mean-field model of the two-species nematic host/chiral dopant mixture is proposed, and it is shown that Oseen-Frank's elastic free energy for cholesteric liquid crystals can be obtained in a simultaneously large-domain and dilute-dopant asymptotic regime. By techniques of $Γ$-convergence, it is shown that in the asymptotic limit dopant-dopant interactions are negligable, the Frank constants and nematic host order parameter are unperturbed by the presence of dopant, but the mirror asymmetry of the dopant-host interaction leads to a macroscopically twisted ground state. The constant of proportionality between the helical wavenumber and dopant concentration, the {\it helical twisting power} (HTP), can be explicitly found through such an analysis, with a non-linear temperature dependence. Depending on the relative strengths of the host-host and host-dopant interactions, it is shown that HTP may increase or decrease with temperature.

math.AP↗