arXiv subjects

Robert Speck

Publications and source records attributed to Robert Speck.

At least 19 recordsLinked to original sources

arXiv2026

On Distributed Parallelization Strategies for Particle-in-Fourier Schemes

We present and compare distributed parallelization strategies for the particle-in-Fourier (PIF) schemes used in kinetic plasma simulations. The different strategies are i) domain decomposition, where both the particles and Fourier modes are split between the MPI ranks ii) particle decomposition, where only the particles are split between the ranks and each rank carries all the modes, and, iii) space-time decomposition, in which time parallelization based on the parareal algorithm is added on top of the particle decomposition. We describe the different communication patterns involved in each of the strategies, the parameter regimes where they work best, and explain their advantages and disadvantages. We implement the strategies within the open-source, performance portable library IPPL and conduct scaling studies with 3D-3V Landau damping and Penning trap benchmark problems on Alps and JUWELS booster supercomputers. We analyze the dominant component timings in each of the strategies and identify areas for future optimizations.

cs.CE↗

arXiv2025

Error Analysis and Parallel Scaling Study of A Parareal Parallel-in-Time Integration Algorithm for Particle-in-Fourier Schemes

We propose a parareal based time parallelization scheme in the phase-space for the particle-in-Fourier (PIF) discretization of the Vlasov-Poisson system used in kinetic plasma simulations. We use PIF with a coarse tolerance for the nonuniform fast Fourier transforms, or the standard particle-in-cell scheme, combined with temporal coarsening, as coarse propagators. This is different from the typical spatial coarsening of particles and/or Fourier modes for parareal, which are not possible or effective for PIF schemes. We perform an error analysis of the algorithm and verify the results numerically with Landau damping, two-stream instability, and Penning trap test cases in 3D-3V. We also implement the space-time parallelization of the PIF schemes in the open-source, performance-portable library IPPL and conduct scaling studies up to 1536 A100 GPUs on the JUWELS booster supercomputer. The space-time parallelization utilizing the parareal algorithm for the time parallelization provides up to $4-6$ times speedup compared to spatial parallelization alone and achieves a push rate of around 1 billion particles per second for the benchmark plasma mini-apps considered.

math.NA↗

arXiv2025

Resilience Against Soft Faults through Adaptivity in Spectral Deferred Correction

As supercomputers grow in hardware complexity, their susceptibility to faults increases and measures need to be taken to ensure the correctness of results. Some numerical algorithms have certain characteristics that allow them to recover from some types of faults. It has been demonstrated that adaptive Runge-Kutta methods provide resilience against transient faults without adding computational cost. Using recent advances in adaptive step size selection for spectral deferred correction (SDC), an iterative numerical time stepping scheme that can produce methods of arbitrary order, we show that adaptive SDC can also detect and correct transient faults. Its performance is found to be comparable to that of the dedicated resilience strategy Hot Rod.

cs.DC↗

arXiv2024

Adaptive time step selection for Spectral Deferred Correction

Spectral Deferred Correction (SDC) is an iterative method for the numerical solution of ordinary differential equations. It works by refining the numerical solution for an initial value problem by approximately solving differential equations for the error, and can be interpreted as a preconditioned fixed-point iteration for solving the fully implicit collocation problem. We adopt techniques from embedded Runge-Kutta Methods (RKM) to SDC in order to provide a mechanism for adaptive time step size selection and thus increase computational efficiency of SDC. We propose two SDC-specific estimates of the local error that are generic and do not rely on problem specific quantities. We demonstrate a gain in efficiency over standard SDC with fixed step size and compare efficiency favorably against state-of-the-art adaptive RKM.

math.NA↗

arXiv2024

Spectral deferred correction methods for second-order problems

Spectral deferred corrections (SDC) are a class of iterative methods for the numerical solution of ordinary differential equations. SDC can be interpreted as a Picard iteration to solve a fully implicit collocation problem, preconditioned with a low-order method. It has been widely studied for first-order problems, using explicit, implicit or implicit-explicit Euler and other low-order methods as preconditioner. For first-order problems, SDC achieves arbitrary order of accuracy and possesses good stability properties. While numerical results for SDC applied to the second-order Lorentz equations exist, no theoretical results are available for SDC applied to second-order problems. We present an analysis of the convergence and stability properties of SDC using velocity-Verlet as the base method for general second-order initial value problems. Our analysis proves that the order of convergence depends on whether the force in the system depends on the velocity. We also demonstrate that the SDC iteration is stable under certain conditions. Finally, we show that SDC can be computationally more efficient than a simple Picard iteration or a fourth-order Runge-Kutta-Nyström method.

math.NA↗

arXiv2023

A parallel-in-time collocation method using diagonalization: theory and implementation for linear problems

We present and analyze a parallel implementation of a parallel-in-time collocation method based on $α$-circulant preconditioned Richardson iterations. While many papers explore this family of single-level, time-parallel "all-at-once" integrators from various perspectives, performance results of actual parallel runs are still scarce. This leaves a critical gap, because the efficiency and applicability of any parallel method heavily rely on the actual parallel performance, with only limited guidance from theoretical considerations. Further, challenges like selecting good parameters, finding suitable communication strategies, and performing a fair comparison to sequential time-stepping methods can be easily missed. In this paper, we first extend the original idea of these fixed point iterative approaches based on $α$-circulant preconditioners to high-order collocation methods, adding yet another level of parallelization in time "across the method". We derive an adaptive strategy to select a new $α$-circulant preconditioner for each iteration during runtime for balancing convergence rates, round-off errors, and inexactness of inner system solves for the individual time-steps. After addressing these more theoretical challenges, we present an open-source space- and time-parallel implementation and evaluate its performance for two different test problems.

math.NA↗

arXiv2021

Convergence analysis of multi-level spectral deferred corrections

The spectral deferred correction (SDC) method is class of iterative solvers for ordinary differential equations (ODEs). It can be interpreted as a preconditioned Picard iteration for the collocation problem. The convergence of this method is well-known, for suitable problems it gains one order per iteration up to the order of the quadrature method of the collocation problem provided. This appealing feature enables an easy creation of flexible, high-order accurate methods for ODEs. A variation of SDC are multi-level spectral deferred corrections (MLSDC). Here, iterations are performed on a hierarchy of levels and an FAS correction term, as in nonlinear multigrid methods, couples solutions on different levels. While there are several numerical examples which show its capabilities and efficiency, a theoretical convergence proof is still missing. This paper addresses this issue. A proof of the convergence of MLSDC, including the determination of the convergence rate in the time-step size, will be given and the results of the theoretical analysis will be numerically demonstrated. It turns out that there are restrictions for the advantages of this method over SDC regarding the convergence rate.

math.NA↗

arXiv2021

The Parallel Full Approximation Scheme in Space and Time for a Parabolic Finite Element Problem

The parallel full approximation scheme in space and time (PFASST) is a parallel-in-time integrator that allows to integrate multiple time-steps simultaneously. It has been shown to extend scaling limits of spatial parallelization strategies when coupled with finite differences, spectral discretizations, or particle methods. In this paper we show how to use PFASST together with a finite element discretization in space. While seemingly straightforward, the appearance of the mass matrix and the need to restrict iterates as well as residuals in space makes this task slightly more intricate. We derive the PFASST algorithm with mass matrices and appropriate prolongation and restriction operators and show numerically that PFASST can, after some initial iterations, gain two orders of accuracy per iteration.

math.NA↗

arXiv2021

Twelve Ways To Fool The Masses When Giving Parallel-In-Time Results

Getting good speedup -- let alone high parallel efficiency -- for parallel-in-time (PinT) integration examples can be frustratingly difficult. The high complexity and large number of parameters in PinT methods can easily (and unintentionally) lead to numerical experiments that overestimate the algorithm's performance. In the tradition of Bailey's article "Twelve ways to fool the masses when giving performance results on parallel computers", we discuss and demonstrate pitfalls to avoid when evaluating performance of PinT methods. Despite being written in a light-hearted tone, this paper is intended to raise awareness that there are many ways to unintentionally fool yourself and others and that by avoiding these fallacies more meaningful PinT performance results can be obtained.

math.NA↗

arXiv2020

Resiliency in Numerical Algorithm Design for Extreme Scale Simulations

This work is based on the seminar titled ``Resiliency in Numerical Algorithm Design for Extreme Scale Simulations'' held March 1-6, 2020 at Schloss Dagstuhl, that was attended by all the authors. Naive versions of conventional resilience techniques will not scale to the exascale regime: with a main memory footprint of tens of Petabytes, synchronously writing checkpoint data all the way to background storage at frequent intervals will create intolerable overheads in runtime and energy consumption. Forecasts show that the mean time between failures could be lower than the time to recover from such a checkpoint, so that large calculations at scale might not make any progress if robust alternatives are not investigated. More advanced resilience techniques must be devised. The key may lie in exploiting both advanced system features as well as specific application knowledge. Research will face two essential questions: (1) what are the reliability requirements for a particular computation and (2) how do we best design the algorithms and software to meet these requirements? One avenue would be to refine and improve on system- or application-level checkpointing and rollback strategies in the case an error is detected. Developers might use fault notification interfaces and flexible runtime systems to respond to node failures in an application-dependent fashion. Novel numerical algorithms or more stochastic computational approaches may be required to meet accuracy requirements in the face of undetectable soft errors. The goal of this Dagstuhl Seminar was to bring together a diverse group of scientists with expertise in exascale computing to discuss novel ways to make applications resilient against detected and undetected faults. In particular, participants explored the role that algorithms and applications play in the holistic approach needed to tackle this challenge.

cs.DC↗

arXiv2020

Using performance analysis tools for parallel-in-time integrators -- Does my time-parallel code do what I think it does?

While many ideas and proofs of concept for parallel-in-time integration methods exists, the number of large-scale, accessible time-parallel codes is rather small. This is often due to the apparent or subtle complexity of the algorithms and the many pitfalls awaiting developers of parallel numerical software. One example of such a time-parallel code is pySDC, which implements, among others, the parallel full approximation scheme in space and time (PFASST). Inspired by nonlinear multigrid ideas, PFASST allows to integrate multiple time-steps simultaneously using a space-time hierarchy of spectral deferred corrections. In this paper we demonstrate the application of performance analysis tools to the PFASST implementation pySDC. Tracing the path we took for this work, we highlight the obstacles encountered, describe remedies and explain the sometimes surprising findings made possible by the tools. Although focusing only on a single implementation of a particular parallel-in-time integrator, we hope that our results and in particular the way we obtained them are a blueprint for other time-parallel codes.

cs.PF↗

arXiv2020

Time-parallel simulation of the Schrödinger Equation

The numerical simulation of the time-dependent Schrödinger equation for quantum systems is a very active research topic. Yet, resolving the solution sufficiently in space and time is challenging and mandates the use of modern high-performance computing systems. While classical parallelization techniques in space can reduce the runtime per time-step, novel parallel-in-time integrators expose parallelism in the temporal domain. They work, however, not very well for wave-type problems such as the Schrödinger equation. One notable exception is the rational approximation of exponential integrators. In this paper we derive an efficient variant of this approach suitable for the complex-valued Schrödinger equation. Using the Faber-Carathéodory-Fejér approximation, this variant is already a fast serial and in particular an efficient time-parallel integrator. It can be used to augment classical parallelization in space and we show the efficiency and effectiveness of our method along the lines of two challenging, realistic examples.

math.NA↗

arXiv2019

PFASST-ER: Combining the Parallel Full Approximation Scheme in Space and Time with parallelization across the method

To extend prevailing scaling limits when solving time-dependent partial differential equations, the parallel full approximation scheme in space and time (PFASST) has been shown to be a promising parallel-in-time integrator. Similar to a space-time multigrid, PFASST is able to compute multiple time-steps simultaneously and is therefore in particular suitable for large-scale applications on high performance computing systems. In this work we couple PFASST with a parallel spectral deferred correction (SDC) method, forming an unprecedented doubly time-parallel integrator. While PFASST provides global, large-scale "parallelization across the step", the inner parallel SDC method allows to integrate each individual time-step "parallel across the method" using a diagonalized local Quasi-Newton solver. This new method, which we call "PFASST with Enhanced concuRrency" (PFASST-ER), therefore exposes even more temporal parallelism. For two challenging nonlinear reaction-diffusion problems, we show that PFASST-ER works more efficiently than the classical variants of PFASST and can be used to run parallel-in-time beyond the number of time-steps.

cs.MS↗

arXiv2018

pySDC - Prototyping spectral deferred corrections

In this paper we present the Python framework pySDC for solving collocation problems with spectral deferred correction methods (SDC) and their time-parallel variant PFASST, the parallel full approximation scheme in space and time. pySDC features many implementations of SDC and PFASST, from simple implicit time-stepping to high-order implicit-explicit or multi-implicit splitting and multi-level spectral deferred corrections. It comes with many different, pre-implemented examples and has seven tutorials to help new users with their first steps. Time-parallelism is implemented either in an emulated way for debugging and prototyping as well as using MPI for benchmarking. The code is fully documented and tested using continuous integration, including most results of previous publications. Here, we describe the structure of the code by taking two different perspectives: the user's and the developer's perspective. While the first sheds light on the front-end, the examples and the tutorials, the second is used to describe the underlying implementation and the data structures. We show three different examples to highlight various aspects of the implementation, the capabilities and the usage of pySDC. Also, couplings to the FEniCS framework and PETSc, the latter including spatial parallelism with MPI, are described.

cs.MS↗

arXiv2018

Asymptotic convergence of the parallel full approximation scheme in space and time for linear problems

For time-dependent partial differential equations, parallel-in-time integration using the "parallel full approximation scheme in space and time" (PFASST) is a promising way to accelerate existing space-parallel approaches beyond their scaling limits. Inspired by the classical Parareal method and multigrid ideas, PFASST allows to integrate multiple time-steps simultaneously using a space-time hierarchy of spectral deferred correction sweeps. While many use cases and benchmarks exist, a solid and reliable mathematical foundation is still missing. Very recently, however, PFASST for linear problems has been identified as multigrid method and in this paper, we will use this multigrid formulation and in particular PFASST's iteration matrix to show that in the non-stiff as well as in the stiff limit PFASST indeed is a convergent iterative method. We will provide upper bounds for the spectral radius of the iteration matrix and investigate how PFASST performs for increasing numbers of parallel time-steps. Finally, we will demonstrate that the results obtained here indeed relate to actual PFASST runs.

math.NA↗

arXiv2017

Parallelizing spectral deferred corrections across the method

In this paper we present two strategies to enable "parallelization across the method" for spectral deferred corrections (SDC). Using standard low-order time-stepping methods in an iterative fashion, SDC can be seen as preconditioned Picard iteration for the collocation problem. Typically, a serial Gauss-Seidel-like preconditioner is used, computing updates for each collocation node one by one. The goal of this paper is to show how this process can be parallelized, so that all collocation nodes are updated simultaneously. The first strategy aims at finding parallel preconditioners for the Picard iteration and we test three choices using four different test problems. For the second strategy we diagonalize the quadrature matrix of the collocation problem directly. In order to integrate non-linear problems we employ simplified and inexact Newton methods. Here, we estimate the speed of convergence depending on the time-step size and verify our results using a non-linear diffusion problem.

math.NA↗

arXiv2016

Spectral deferred corrections with fast-wave slow-wave splitting

The paper investigates a variant of semi-implicit spectral deferred corrections (SISDC) in which the stiff, fast dynamics correspond to fast propagating waves ("fast-wave slow-wave problem"). We show that for a scalar test problem with two imaginary eigenvalues $i λ_{fast}$, $i λ_{slow}$, having $Δt \left(\left| λ_{fast} \right| + \left| λ_{slow} \right| \right) < 1$ is sufficient for the fast-wave slow-wave SDC (FWSW-SDC) iteration to converge and that in the limit of infinitely fast waves the convergence rate of the non-split version is retained. Stability function and discrete dispersion relation are derived and show that the method is stable for essentially arbitrary fast-wave CFL numbers as long as the slow dynamics are resolved. The method causes little numerical diffusion and its semi-discrete phase speed is accurate also for large wave number modes. Performance is studied for an acoustic-advection problem and for the linearised Boussinesq equations, describing compressible, stratified flow. FWSW-SDC is compared to a diagonally implicit Runge-Kutta (DIRK) and IMEX Runge-Kutta (IMEX) method and found to be competitive in terms of both accuracy and cost.

math.NA↗

arXiv2016

Toward fault-tolerant parallel-in-time integration with PFASST

We introduce and analyze different strategies for the parallel-in-time integration method PFASST to recover from hard faults and subsequent data loss. Since PFASST stores solutions at multiple time steps on different processors, information from adjacent steps can be used to recover after a processor has failed. PFASST's multi-level hierarchy allows to use the coarse level for correcting the reconstructed solution, which can help to minimize overhead. A theoretical model is devised linking overhead to the number of additional PFASST iterations required for convergence after a fault. The potential efficiency of different strategies is assessed in terms of required additional iterations for examples of diffusive and advective type.

cs.DC↗