arXiv subjects

Tim Warburton

Publications and source records attributed to Tim Warburton.

At least 19 recordsLinked to original sources

arXiv2026

A High-Order Arbitrary Lagrangian-Eulerian Discontinuous Galerkin Method for the Boltzmann Equation in Nearly Incompressible Flows

We propose the arbitrary Lagrangian-Eulerian (ALE) form of the Galerkin-Boltzmann formulation for the simulation of nearly incompressible flows with moving boundaries. The continuous Boltzmann equations are mapped to a reference state to compensate the mesh motion with an advection term. The resulting system is discretized in space using the discontinuous Galerkin method on unstructured meshes. A semi-analytic Runge-Kutta time discretization is used to overcome the stiffness introduced by the continuous Boltzmann equations. The well-known geometric conservation law is shown to be satisfied by the time and space discretizations and consistent update of geometric factors of the discretization. The implementation is on the GPU accelerated kernel library libParanumal and validated by a free stream preservation and moving Taylor-Green vortex test cases. Then, the capabilities are shown using a plunging symmetric airfoil in two-dimensions and moving carangiform fish in three-dimensions using perfectly matched layers.

physics.flu-dyn↗

arXiv2026

Implicit Large Eddy Simulation of Nearly Incompressible Flows with a Discontinuous Galerkin-Boltzmann Formulation

We present a high-order implicit large eddy simulation (ILES) approach for simulating flows at the nearly incompressible regime. Our methodology based on utilization of a nodal discontinuous Galerkin (DG) discretization of the Boltzmann equations. The compactness and low-dissipative nature of the discontinuous Galerkin method are leveraged to mimic traditional large eddy simulations with subgrid-scale models. One of the key requirements of ILES is to provide dissipation only within a narrow band of high wavenumbers. This is validated through numerical experiments on the Taylor-Green Vortex problem in detail at a Reynolds number where varying scales of coherent turbulent structures are present. Furthermore, the approach is validated for external aerodynamic configurations by simulating the flow over a sphere at a Reynolds number of $Re=3700$, capturing the laminar-turbulent transition and the complex multiscale vortex dynamics characteristic of this regime. The results demonstrate the capability of the high-order DG-Boltzmann formulation to accurately capture transitional and turbulent flow features without the use of explicit sub-grid scale modeling, highlighting its potential as a robust and physically consistent framework for ILES of nearly incompressible turbulent flows.

physics.flu-dyn↗

arXiv2025

An Adaptive Mixed Precision and Dynamically Scaled Preconditioned Conjugate Gradient Algorithm

We propose an adaptive mixed precision and dynamically scaled preconditioned conjugate gradient algorithm (AMP-PCG). It dynamically adjusts the precision for storing vectors and computing, exploiting low precision when appropriate, while maintaining a convergence rate and accuracy comparable to that of double precision PCG. Our mixed precision strategy consists of three main components: (1) The residual and matrix-vector product are initially computed in double precision, and the algorithm switches these to single precision based on the chosen convergence tolerance and an estimate of the residual gap. (2) Depending on the eigenvalue distribution, the preconditioned residual and search direction are either in half precision throughout the iterations or initially in double precision and then stepwise reduced to single and half precision. (3) A dynamically scaled residual is used at every iteration to mitigate underflow in half precision. We provide theoretical support for our estimates and we demonstrate the effectiveness of AMP-PCG through numerical experiments, highlighting both its robustness and the significant performance gains (1.63x speedup) achieved compared to double precision PCG on a GPU.

math.NA↗

arXiv2024

Exascale Simulations of Fusion and Fission Systems

We discuss pioneering heat and fluid flow simulations of fusion and fission energy systems with NekRS on exascale computing facilities, including Frontier and Aurora. The Argonne-based code, NekRS, is a highly-performant open-source code for the simulation of incompressible and low-Mach fluid flow, heat transfer, and combustion with a particular focus on turbulent flows in complex domains. It is based on rapidly convergent high-order spectral element discretizations that feature minimal numerical dissipation and dispersion. State-of-the-art multilevel preconditioners, efficient high-order time-splitting methods, and runtime-adaptive communication strategies are built on a fast OCCA-based kernel library, libParanumal, to provide scalability and portability across the spectrum of current and future high-performance computing platforms. On Frontier, Nek5000/RS has achieved an unprecedented milestone in breaching over 1 trillion degrees of freedom with the spectral element methods for the simulation of the CHIMERA fusion technology testing platform. We also demonstrate for the first time the use of high-order overset grids at scale.

cs.CE↗

arXiv2024

A stable decoupled perfectly matched layer for the 3D wave equation using the nodal discontinuous Galerkin method

In outdoor acoustics, the calculations of sound propagating in air can be computationally heavy if the domain is chosen large enough to fulfil the Sommerfeld radiation condition. By strategically truncating the computational domain with a efficient boundary treatment, the computational cost is lowered. One commonly used boundary treatment is the perfectly matched layer (PML) that dampens outgoing waves without polluting the computed solution in the inner domain. The purpose of this study is to propose and assess a new perfectly matched layer formulation for the 3D acoustic wave equation, using the nodal discontinuous Galerkin finite element method. The formulation is based on an efficient PML formulation that can be decoupled to further increase the computational efficiency and guarantee stability without sacrificing accuracy. This decoupled PML formulation is demonstrated to be long-time stable and an optimization procedure of the damping functions is proposed to enhance the performance of the formulation.

math.NA↗

arXiv2023

Stopping Criteria for the Conjugate Gradient Algorithm in High-Order Finite Element Methods

We consider stopping criteria that balance algebraic and discretization errors for the conjugate gradient algorithm applied to high-order finite element discretizations of Poisson problems. Firstly, we introduce a new stopping criterion that suggests stopping when the norm of the linear system residual is less than a small fraction of an error indicator derived directly from the residual. This indicator shares the same mesh size and polynomial degree scaling as the norm of the residual, resulting in a robust criterion regardless of the mesh size, the polynomial degree, and the shape regularity of the mesh. Secondly, for solving Poisson problems with highly variable piecewise constant coefficients, we introduce a subdomain-based criterion that recommends stopping when the norm of the linear system residual restricted to each subdomain is smaller than the corresponding indicator also restricted to that subdomain. Reliability and efficiency theorems for the first criterion are established. Numerical experiments, including tests with highly variable piecewise constant coefficients and a GPU-accelerated three-dimensional elliptic solver, demonstrate that the proposed criteria efficiently avoid both premature termination and over-solving.

math.NA↗

arXiv2022

On the entropy projection and the robustness of high order entropy stable discontinuous Galerkin schemes for under-resolved flows

High order entropy stable schemes provide improved robustness for computational simulations of fluid flows. However, additional stabilization and positivity preserving limiting can still be required for variable-density flows with under-resolved features. We demonstrate numerically that entropy stable DG methods which incorporate an "entropy projection" are less likely to require additional limiting to retain positivity for certain types of flows. We conclude by investigating potential explanations for this observed improvement in robustness.

math.NA↗

arXiv2022

HipBone: A performance-portable GPU-accelerated C++ version of the NekBone benchmark

We present hipBone, an open source performance-portable proxy application for the Nek5000 (and NekRS) CFD applications. HipBone is a fully GPU-accelerated C++ implementation of the original NekBone CPU proxy application with several novel algorithmic and implementation improvements which optimize its performance on modern fine-grain parallel GPU accelerators. Our optimizations include a conversion to store the degrees of freedom of the problem in assembled form in order to reduce the amount of data moved during the main iteration and a portable implementation of the main Poisson operator kernel. We demonstrate near-roofline performance of the operator kernel on three different modern GPU accelerators from two different vendors. We present a novel algorithm for splitting the application of the Poisson operator on GPUs which aggressively hides MPI communication required for both halo exchange and assembly. Our implementation of nearest-neighbor MPI communication then leverages several different routing algorithms and GPU-Direct RDMA capabilities, when available, which improves scalability of the benchmark. We demonstrate the performance of hipBone on three different clusters housed at Oak Ridge National Laboratory, namely the Summit supercomputer and the Frontier early-access clusters, Spock and Crusher. Our tests demonstrate both portability across different clusters and very good scaling efficiency, especially on large problems.

cs.DC↗

arXiv2021

Highly Optimized Full-Core Reactor Simulations on Summit

Nek5000/RS is a highly-performant open-source spectral element code for simulation of incompressible and low-Mach fluid flow, heat transfer, and combustion with a particular focus on turbulent flows in complex domains. It is based on high-order discretizations that realize the same (or lower) cost per gridpoint as traditional low-order methods. State-of-the-art multilevel preconditioners, efficient high-order time-splitting methods, and runtime-adaptive communication strategies are built on a fast OCCA-based kernel library, libParanumal, to provide scalability and portability across the spectrum of current and future high-performance computing platforms. On Summit, Nek5000/RS has recently achieved an milestone in the simulation of nuclear reactors: the first full-core computational fluid dynamics simulations of reactor cores, including pebble beds with > 350,000 pebbles and 98M elements advanced in less than 0.25 seconds per Navier-Stokes timestep. With carefully tuned algorithms, it is possible to simulate a single flow-through time for a full reactor core in less than six hours on all of Summit.

physics.comp-ph↗

arXiv2021

GPU Algorithms for Efficient Exascale Discretizations

In this paper we describe the research and development activities in the Center for Efficient Exascale Discretization within the US Exascale Computing Project, targeting state-of-the-art high-order finite-element algorithms for high-order applications on GPU-accelerated platforms. We discuss the GPU developments in several components of the CEED software stack, including the libCEED, MAGMA, MFEM, libParanumal, and Nek projects. We report performance and capability improvements in several CEED-enabled applications on both NVIDIA and AMD GPU systems.

cs.DC↗

arXiv2021

Efficient Exascale Discretizations: High-Order Finite Element Methods

Efficient exploitation of exascale architectures requires rethinking of the numerical algorithms used in many large-scale applications. These architectures favor algorithms that expose ultra fine-grain parallelism and maximize the ratio of floating point operations to energy intensive data movement. One of the few viable approaches to achieve high efficiency in the area of PDE discretizations on unstructured grids is to use matrix-free/partially-assembled high-order finite element methods, since these methods can increase the accuracy and/or lower the computational time due to reduced data motion. In this paper we provide an overview of the research and development activities in the Center for Efficient Exascale Discretizations (CEED), a co-design center in the Exascale Computing Project that is focused on the development of next-generation discretization software and algorithms to enable a wide range of finite element applications to run efficiently on future hardware. CEED is a research partnership involving more than 30 computational scientists from two US national labs and five universities, including members of the Nek5000, MFEM, MAGMA and PETSc projects. We discuss the CEED co-design activities based on targeted benchmarks, miniapps and discretization libraries and our work on performance optimizations for large-scale GPU architectures. We also provide a broad overview of research and development activities in areas such as unstructured adaptive mesh refinement algorithms, matrix-free linear solvers, high-order data visualization, and list examples of collaborations with several ECP and external applications.

cs.DC↗

arXiv2021

A Local Discontinuous Galerkin Level Set Reinitialization with Subcell Stabilization on Unstructured Meshes

In this paper we consider a level set reinitialization technique based on a high-order, local discontinuous Galerkin method on unstructured triangular meshes. A finite volume based subcell stabilization is used to improve the nonlinear stability of the method. Instead of the standard hyperbolic level set reinitialization, the flow of time Eikonal equation is discretized to construct an approximate signed distance function. Using the Eikonal equation removes the regularization parameter in the standard approach which allows more predictable behavior and faster convergence speeds around the interface. This makes our approach very efficient especially for banded level set formulations. A set of numerical experiments including both smooth and non-smooth interfaces indicate that the method experimentally achieves design order accuracy.

math.NA↗

arXiv2021

NekRS, a GPU-Accelerated Spectral Element Navier-Stokes Solver

The development of NekRS, a GPU-oriented thermal-fluids simulation code based on the spectral element method (SEM) is described. For performance portability, the code is based on the open concurrent compute abstraction and leverages scalable developments in the SEM code Nek5000 and in libParanumal, which is a library of high-performance kernels for high-order discretizations and PDE-based miniapps. Critical performance sections of the Navier-Stokes time advancement are addressed. Performance results on several platforms are presented, including scaling to 27,648 V100s on OLCF Summit, for calculations of up to 60B gridpoints.

cs.PF↗

arXiv2020

Entropy stable modal discontinuous Galerkin schemes and wall boundary conditions for the compressible Navier-Stokes equations

Entropy stable schemes ensure that physically meaningful numerical solutions also satisfy a semi-discrete entropy inequality under appropriate boundary conditions. In this work, we describe a discretization of viscous terms in the compressible Navier-Stokes equations which enables a simple and explicit imposition of entropy stable no-slip (adiabatic and isothermal) and reflective (symmetry) wall boundary conditions for discontinuous Galerkin (DG) discretizations. Numerical results confirm the robustness and accuracy of the proposed approaches.

math.NA↗

arXiv2020

Initial Guesses for Sequences of Linear Systems in a GPU-Accelerated Incompressible Flow Solver

We consider several methods for generating initial guesses when iteratively solving sequences of linear systems, showing that they can be implemented efficiently in GPU-accelerated PDE solvers, specifically solvers for incompressible flow. We propose new initial guess methods based on stabilized polynomial extrapolation and compare them to the projection method of Fischer [15], showing that they are generally competitive with projection schemes despite requiring only half the storage and performing considerably less data movement and communication. Our implementations of these algorithms are freely available as part of the libParanumal collection of GPU-accelerated flow solvers.

math.NA↗

arXiv2020

Portable high-order finite element kernels I: Streaming Operations

This paper is devoted to the development of highly efficient kernels performing vector operations relevant in linear system solvers. In particular, we focus on the low arithmetic intensity operations (i.e., streaming operations) performed within the conjugate gradient iterative method, using the parameters specified in the CEED benchmark problems for high-order hexahedral finite elements. We propose a suite of new Benchmark Streaming tests to focus on the distinct streaming operations which must be performed. We implemented these new tests using the OCCA abstraction framework to demonstrate portability of these streaming operations on different GPU architectures, and propose a simple performance model for such kernels which can accurately capture data movement rates as well as kernel launch costs.

cs.MS↗

arXiv2020

Scalability of High-Performance PDE Solvers

Performance tests and analyses are critical to effective HPC software development and are central components in the design and implementation of computational algorithms for achieving faster simulations on existing and future computing architectures for large-scale application problems. In this paper, we explore performance and space-time trade-offs for important compute-intensive kernels of large-scale numerical solvers for PDEs that govern a wide range of physical applications. We consider a sequence of PDE- motivated bake-off problems designed to establish best practices for efficient high-order simulations across a variety of codes and platforms. We measure peak performance (degrees of freedom per second) on a fixed number of nodes and identify effective code optimization strategies for each architecture. In addition to peak performance, we identify the minimum time to solution at 80% parallel efficiency. The performance analysis is based on spectral and p-type finite elements but is equally applicable to a broad spectrum of numerical PDE discretizations, including finite difference, finite volume, and h-type finite elements.

cs.PF↗

arXiv2018

Leapfrog time-stepping for Hermite methods

We introduce Hermite-leapfrog methods for first order wave systems. The new Hermite-leapfrog methods pair leapfrog time-stepping with the Hermite methods of Goodrich and co-authors. The new schemes stagger field variables in both time and space and are high-order accurate. We provide a detailed description of the method and demonstrate that the method conserves variable quantities in one-space dimension. Higher dimensional versions of the method are constructed via a tensor product construction. Numerical evidence and rigorous analysis in one space dimension establish stability and high-order convergence. Experiments demonstrating efficient implementations on a graphics processing unit are also presented.

math.NA↗