arXiv subjects

Scott MacLachlan

Publications and source records attributed to Scott MacLachlan.

At least 19 recordsLinked to original sources

arXiv2025

Unstructured to structured: geometric multigrid on complex geometries via domain remapping

For domains that are easily represented by structured meshes, robust geometric multigrid solvers can quickly provide the numerical solution to many discretized elliptic PDEs. However, for complicated domains with unstructured meshes, constructing suitable hierarchies of meshes becomes challenging. We propose a framework for mapping computations from such complex domains to regular computational domains via diffeomorphisms, enabling the use of robust geometric-style multigrid. This mapping facilitates regular memory accesses during solves, improving efficiency and scalability, especially on massively parallel processors such as GPUs. Moreover, we show that the diffeomorphic mapping itself may be approximately learned using an invertible neural network, facilitating automated application to geometries where no analytic mapping is readily available.

math.NA↗

arXiv2025

Improving patch selection for monolithic multigrid solvers for high-order {T}aylor-{H}ood discretizations

Numerical simulation of incompressible fluid flows has been an active topic of research in Scientific Computing for many years, with many contributions to both discretizations and linear and nonlinear solvers. In this work, we propose an improved relaxation scheme for higher-order Taylor-Hood discretizations of the incompressible Stokes and Navier-Stokes equations, demonstrating its efficiency within monolithic multigrid preconditioners for the linear(ized) equations. The key to this improvement is an improved patch construction for Vanka-style relaxation introducing, for the first time, overlap in the pressure degrees of freedom within the patches. Numerical results demonstrate significant improvement in both multigrid iterations and time-to-solution for the linear Stokes case, on both triangular and quadrilateral meshes. For the nonlinear Navier-Stokes case, we show similar improvements, including in the number of nonlinear iterations needed in an inexact Newton method. These improvements enable three-dimensional numerical simulations, which also presented.

math.NA↗

arXiv2024

Unique continuation principles for finite-element discretizations of the Laplacian

Unique continuation principles are fundamental properties of elliptic partial differential equations, giving conditions that guarantee that the solution to an elliptic equation must be uniformly zero. Since finite-element discretizations are a natural tool to help gain understanding into elliptic equations, it is natural to ask if such principles also hold at the discrete level. In this work, we prove a version of the unique continuation principle for piecewise-linear and -bilinear finite-element discretizations of the Laplacian eigenvalue problem on polygonal domains in $\mathbb{R}^2$. Namely, we show that any solution to the discretized equation $-\Delta u = \lambda u$ with vanishing Dirichlet and Neumann traces must be identically zero under certain geometric and topological assumptions on the resulting triangulation. We also provide a counterexample, showing that a nonzero \emph{inner solution} exists when the topological assumptions are not satisfied. Finally, we give an application to an eigenvalue interlacing problem, where the space of inner solutions makes an explicit appearance.

math.NA↗

arXiv2024

Achieving $h$- and $p$-robust monolithic multigrid solvers for the Stokes equations

The numerical analysis of higher-order mixed finite-element discretizations for saddle-point problems, such as the Stokes equations, has been well-studied in recent years. While the theory and practice of such discretizations is now well-understood, the same cannot be said for efficient preconditioners for solving the resulting linear (or linearized) systems of equations. In this work, we propose and study variants of the well-known Vanka relaxation scheme that lead to effective geometric multigrid preconditioners for both the conforming Taylor-Hood discretizations and non-conforming ${\bf H}(\text{div})$-$L^2$ discretizations of the Stokes equations. Numerical results demonstrate robust performance with respect to FGMRES iteration counts for increasing polynomial order for some of the considered discretizations, and expose open questions about stopping tolerances for effectively preconditioned iterations at high polynomial order.

math.NA↗

arXiv2024

Space-time waveform relaxation multigrid for Navier-Stokes

Space-time finite-element discretizations are well-developed in many areas of science and engineering, but much work remains within the development of specialized solvers for the resulting linear and nonlinear systems. In this work, we consider the all-at-once solution of the discretized Navier-Stokes equations over a space-time domain using waveform relaxation multigrid methods. In particular, we show how to extend the efficient spatial multigrid relaxation methods from [37] to a waveform relaxation method, and demonstrate the efficiency of the resulting monolithic Newton-Krylov-multigrid solver. Numerical results demonstrate the scalability of the solver for varying discretization order and physical parameters.

math.NA↗

arXiv2024

Monolithic Multigrid Preconditioners for High-Order Discretizations of Stokes Equations

This work introduces and assesses the efficiency of a monolithic $ph$MG multigrid framework designed for high-order discretizations of stationary Stokes systems using Taylor-Hood and Scott-Vogelius elements. The proposed approach integrates coarsening in both approximation order ($p$) and mesh resolution ($h$), to address the computational and memory efficiency challenges that are often encountered in conventional high-order numerical simulations. Our numerical results reveal that $ph$MG offers significant improvements over traditional spatial-coarsening-only multigrid ($h$MG) techniques for problems discretized with Taylor-Hood elements across a variety of problem sizes and discretization orders. In particular, the $ph$MG method exhibits superior performance in reducing setup and solve times, particularly when dealing with higher discretization orders and unstructured problem domains. For Scott-Vogelius discretizations, while monolithic $ph$MG delivers low iteration counts and competitive solve phase timings, it exhibits a discernibly slower setup phase when compared to a multilevel (non-monolithic) full-block-factorization (FBF) preconditioner where $ph$MG is employed only for the velocity unknowns. This is primarily due to the setup costs of the larger mixed-field relaxation patches with monolithic $ph$MG versus the patch setup costs with a single unknown type for FBF.

math.NA↗

arXiv2024

Exploiting mesh structure to improve multigrid performance for saddle point problems

In recent years, solvers for finite-element discretizations of linear or linearized saddle-point problems, like the Stokes and Oseen equations, have become well established. There are two main classes of preconditioners for such systems: those based on block-factorization approach and those based on monolithic multigrid. Both classes of preconditioners have several critical choices to be made in their composition, such as the selection of a suitable relaxation scheme for monolithic multigrid. From existing studies, some insight can be gained as to what options are preferable in low-performance computing settings, but there are very few fair comparisons of these approaches in the literature, particularly for modern architectures, such as GPUs. In this paper, we perform a comparison between a block-triangular preconditioner and a monolithic multigrid method with the three most common choices of relaxation scheme - Braess-Sarazin, Vanka, and Schur-Uzawa. We develop a performant Vanka relaxation algorithm for structured-grid discretizations, which takes advantage of memory efficiencies in this setting. We detail the behavior of the various CUDA kernels for the multigrid relaxation schemes and evaluate their individual arithmetic intensity, performance, and runtime. Running a preconditioned FGMRES solver for the Stokes equations with these preconditioners allows us to compare their efficiency in a practical setting. We show monolithic multigrid can outperform block-triangular preconditioning, and that using Vanka or Braess-Sarazin relaxation is most efficient. Even though multigrid with Vanka relaxation exhibits reduced performance on the CPU (up to $100\%$ slower than Braess-Sarazin), it is able to outperform Braess-Sarazin by more than $20\%$ on the GPU, making it a competitive algorithm, especially given the high amount of algorithmic tuning needed for effective Braess-Sarazin relaxation.

math.NA↗

arXiv2023

Monolithic Algebraic Multigrid Preconditioners for the Stokes Equations

We investigate a novel monolithic algebraic multigrid (AMG) preconditioner for the Taylor-Hood ($\pmb{\mathbb{P}}_2/\mathbb{P}_1$) and Scott-Vogelius ($\pmb{\mathbb{P}}_2/\mathbb{P}_1^{disc}$) discretizations of the Stokes equations. The algorithm is based on the use of the lower-order $\pmb{\mathbb{P}}_1\text{iso}\kern1pt\pmb{\mathbb{P}}_2/\mathbb{P}_1$ operator within a defect-correction setting, in combination with AMG construction of interpolation operators for velocities and pressures. The preconditioning framework is primarily algebraic, though the $\pmb{\mathbb{P}}_1\text{iso}\kern1pt\pmb{\mathbb{P}}_2/\mathbb{P}_1$ operator must be provided. We investigate two relaxation strategies in this setting. Specifically, a novel block factorization approach is devised for Vanka patch systems, which significantly reduces storage requirements and computational overhead, and a Chebyshev adaptation of the LSC-DGS relaxation is developed to improve parallelism. The preconditioner demonstrates robust performance across a variety of 2D and 3D Stokes problems, often matching or exceeding the effectiveness of an inexact block-triangular (or Uzawa) preconditioner, especially in challenging scenarios such as elongated-domain problems.

math.NA↗

arXiv2023

A positivity-preserving unigrid method for elliptic PDEs

While constraints arise naturally in many physical models, their treatment in mathematical and numerical models varies widely, depending on the nature of the constraint and the availability of simulation tools to enforce it. In this paper, we consider the solution of discretized PDE models that have a natural constraint on the positivity (or non-negativity) of the solution. While discretizations of such models often offer analogous positivity properties on their exact solutions, the use of approximate solution algorithms (and the unavoidable effects of floating -- point arithmetic) often destroy any guarantees that the computed approximate solution will satisfy the (discretized form of the) physical constraints, unless the discrete model is solved to much higher precision than discretization error would dictate. Here, we introduce a class of iterative solution algorithms, based on the unigrid variant of multigrid methods, where such positivity constraints can be preserved throughout the approximate solution process. Numerical results for one- and two-dimensional model problems show both the effectiveness of the approach and the trade-off required to ensure positivity of approximate solutions throughout the solution process.

math.NA↗

arXiv2023

Generalizing Lloyd's algorithm for graph clustering

Clustering is a commonplace problem in many areas of data science, with applications in biology and bioinformatics, understanding chemical structure, image segmentation, building recommender systems, and many more fields. While there are many different clustering variants (based on given distance or graph structure, probability distributions, or data density), we consider here the problem of clustering nodes in a graph, motivated by the problem of aggregating discrete degrees of freedom in multigrid and domain decomposition methods for solving sparse linear systems. Specifically, we consider the challenge of forming balanced clusters in the graph of a sparse matrix for use in algebraic multigrid, although the algorithm has general applicability. Based on an extension of the Bellman-Ford algorithm, we generalize Lloyd's algorithm for partitioning subsets of Rn to balance the number of nodes in each cluster; this is accompanied by a rebalancing algorithm that reduces the overall energy in the system. The algorithm provides control over the number of clusters and leads to "well centered" partitions of the graph. Theoretical results are provided to establish linear complexity and numerical results in the context of algebraic multigrid highlight the benefits of improved clustering.

math.NA↗

arXiv2023

MG-GNN: Multigrid Graph Neural Networks for Learning Multilevel Domain Decomposition Methods

Domain decomposition methods (DDMs) are popular solvers for discretized systems of partial differential equations (PDEs), with one-level and multilevel variants. These solvers rely on several algorithmic and mathematical parameters, prescribing overlap, subdomain boundary conditions, and other properties of the DDM. While some work has been done on optimizing these parameters, it has mostly focused on the one-level setting or special cases such as structured-grid discretizations with regular subdomain construction. In this paper, we propose multigrid graph neural networks (MG-GNN), a novel GNN architecture for learning optimized parameters in two-level DDMs\@. We train MG-GNN using a new unsupervised loss function, enabling effective training on small problems that yields robust performance on unstructured grids that are orders of magnitude larger than those in the training set. We show that MG-GNN outperforms popular hierarchical graph network architectures for this optimization and that our proposed loss function is critical to achieving this improved performance.

cs.LG↗

arXiv2022

Generalizing Reduction-Based Algebraic Multigrid

Algebraic Multigrid (AMG) methods are often robust and effective solvers for solving the large and sparse linear systems that arise from discretized PDEs and other problems, relying on heuristic graph algorithms to achieve their performance. Reduction-based AMG (AMGr) algorithms attempt to formalize these heuristics by providing two-level convergence bounds that depend concretely on properties of the partitioning of the given matrix into its fine- and coarse-grid degrees of freedom. MacLachlan and Saad (SISC 2007) proved that the AMGr method yields provably robust two-level convergence for symmetric and positive-definite matrices that are diagonally dominant, with a convergence factor bounded as a function of a coarsening parameter. However, when applying AMGr algorithms to matrices that are not diagonally dominant, not only do the convergence factor bounds not hold, but measured performance is notably degraded. Here, we present modifications to the classical AMGr algorithm that improve its performance on matrices that are not diagonally dominant, making use of strength of connection, sparse approximate inverse (SPAI) techniques, and interpolation truncation and rescaling, to improve robustness while maintaining control of the algorithmic costs. We present numerical results demonstrating the robustness of this approach for both classical isotropic diffusion problems and for non-diagonally dominant systems coming from anisotropic diffusion.

math.NA↗

arXiv2022

Optimized Sparse Matrix Operations for Reverse Mode Automatic Differentiation

Sparse matrix representations are ubiquitous in computational science and machine learning, leading to significant reductions in compute time, in comparison to dense representation, for problems that have local connectivity. The adoption of sparse representation in leading ML frameworks such as PyTorch is incomplete, however, with support for both automatic differentiation and GPU acceleration missing. In this work, we present an implementation of a CSR-based sparse matrix wrapper for PyTorch with CUDA acceleration for basic matrix operations, as well as automatic differentiability. We also present several applications of the resulting sparse kernels to optimization problems, demonstrating ease of implementation and performance measurements versus their dense counterparts.

cs.LG↗

arXiv2022

Constraint-satisfying Krylov solvers for structure-preserving discretisations

A key consideration in the development of numerical schemes for time-dependent partial differential equations (PDEs) is the ability to preserve certain properties of the continuum solution, such as associated conservation laws or other geometric structures of the solution. There is a long history of the development and analysis of such structure-preserving discretisation schemes, including both proofs that standard schemes have structure-preserving properties and proposals for novel schemes that achieve both high-order accuracy and exact preservation of certain properties of the continuum differential equation. When coupled with implicit time-stepping methods, a major downside to these schemes is that their structure-preserving properties generally rely on exact solution of the (possibly nonlinear) systems of equations defining each time-step in the discrete scheme. For small systems, this is often possible (up to the accuracy of floating-point arithmetic), but it becomes impractical for the large linear systems that arise when considering typical discretisation of space-time PDEs. In this paper, we propose a modification to the standard flexible generalised minimum residual (FGMRES) iteration that enforces selected constraints on approximate numerical solutions. We demonstrate its application to both systems of conservation laws and dissipative systems.

math.NA↗

arXiv2022

Learning Interface Conditions in Domain Decomposition Solvers

Domain decomposition methods are widely used and effective in the approximation of solutions to partial differential equations. Yet the optimal construction of these methods requires tedious analysis and is often available only in simplified, structured-grid settings, limiting their use for more complex problems. In this work, we generalize optimized Schwarz domain decomposition methods to unstructured-grid problems, using Graph Convolutional Neural Networks (GCNNs) and unsupervised learning to learn optimal modifications at subdomain interfaces. A key ingredient in our approach is an improved loss function, enabling effective training on relatively small problems, but robust performance on arbitrarily large problems, with computational cost linear in problem size. The performance of the learned linear solvers is compared with both classical and optimized domain decomposition algorithms, for both structured- and unstructured-grid problems.

cs.LG↗

arXiv2022

Monolithic multigrid for implicit Runge-Kutta discretizations of incompressible fluid flow

Most research on preconditioners for time-dependent PDEs has focused on implicit multi-step or diagonally-implicit multi-stage temporal discretizations. In this paper, we consider monolithic multigrid preconditioners for fully-implicit multi-stage Runge-Kutta (RK) time integration methods. These temporal discretizations have very attractive accuracy and stability properties, but they couple the spatial degrees of freedom across multiple time levels, requiring the solution of very large linear systems. We extend the classical Vanka relaxation scheme to implicit RK discretizations of saddle point problems. We present numerical results for the incompressible Stokes, Navier-Stokes, and resistive magnetohydrodynamics equations, in two and three dimensions, confirming that these relaxation schemes lead to robust and scalable monolithic multigrid methods for a challenging range of incompressible fluid-flow models.

math.NA↗

arXiv2022

Monolithic multigrid for a reduced-quadrature discretization of poroelasticity

Advanced finite-element discretizations and preconditioners for models of poroelasticity have attracted significant attention in recent years. The equations of poroelasticity offer significant challenges in both areas, due to the potentially strong coupling between unknowns in the system, saddle-point structure, and the need to account for wide ranges of parameter values, including limiting behavior such as incompressible elasticity. This paper was motivated by an attempt to develop monolithic multigrid preconditioners for the discretization developed in [48]; we show here why this is a difficult task and, as a result, we modify the discretization in [48] through the use of a reduced quadrature approximation, yielding a more "solver-friendly" discretization. Local Fourier analysis is used to optimize parameters in the resulting monolithic multigrid method, allowing a fair comparison between the performance and costs of methods based on Vanka and Braess-Sarazin relaxation. Numerical results are presented to validate the LFA predictions and demonstrate efficiency of the algorithms. Finally, a comparison to existing block-factorization preconditioners is also given.

math.NA↗

arXiv2022

Optimization-Based Algebraic Multigrid Coarsening Using Reinforcement Learning

Large sparse linear systems of equations are ubiquitous in science and engineering, such as those arising from discretizations of partial differential equations. Algebraic multigrid (AMG) methods are one of the most common methods of solving such linear systems, with an extensive body of underlying mathematical theory. A system of linear equations defines a graph on the set of unknowns and each level of a multigrid solver requires the selection of an appropriate coarse graph along with restriction and interpolation operators that map to and from the coarse representation. The efficiency of the multigrid solver depends critically on this selection and many selection methods have been developed over the years. Recently, it has been demonstrated that it is possible to directly learn the AMG interpolation and restriction operators, given a coarse graph selection. In this paper, we consider the complementary problem of learning to coarsen graphs for a multigrid solver, a necessary step in developing fully learnable AMG methods. We propose a method using a reinforcement learning (RL) agent based on graph neural networks (GNNs), which can learn to perform graph coarsening on small planar training graphs and then be applied to unstructured large planar graphs, assuming bounded node degree. We demonstrate that this method can produce better coarse graphs than existing algorithms, even as the graph size increases and other properties of the graph are varied. We also propose an efficient inference procedure for performing graph coarsening that results in linear time complexity in graph size.

cs.LG↗