arXiv subjects

Sriramkrishnan Muralikrishnan

Publications and source records attributed to Sriramkrishnan Muralikrishnan.

15 recordsLinked to original sources

arXiv2026

A Performance-Portable, Massively Parallel Distributed Nonuniform FFT

The nonuniform fast Fourier transform (NUFFT) enables spectral methods for problems with irregularly spaced samples, with applications in medical imaging, molecular dynamics, and kinetic plasma simulations. Existing implementations are limited to shared-memory execution, restricting problem sizes to what fits on a single node. We present the first distributed, performance-portable NUFFT for heterogeneous supercomputers. Our Kokkos-based implementation runs without modification on NVIDIA and AMD GPUs. We develop multiple spreading and interpolation kernels optimized for different accuracy requirements and architectures. Our spreading kernels match or exceed the single-GPU throughput of the state-of-the-art CUDA-based NUFFT library cuFINUFFT at production particle densities, while our Kokkos-based implementation additionally supports AMD GPUs. Strong scaling experiments on Alps (NVIDIA GH200), JUWELS Booster (NVIDIA A100), and LUMI (AMD MI250X) demonstrate scaling up to 1024 GPUs. At scale, the distributed FFT is a significant part of the total runtime, making higher NUFFT accuracy less expensive. We apply the method to massively parallel Particle-in-Fourier simulations of Landau damping with up to $1024^3$ Fourier modes and 8.6 billion particles on Alps, JUWELS, and LUMI, demonstrating that distributed NUFFTs enable kinetic plasma simulations at resolutions previously inaccessible to spectral particle methods.

cs.CE↗

arXiv2026

On Distributed Parallelization Strategies for Particle-in-Fourier Schemes

We present and compare distributed parallelization strategies for the particle-in-Fourier (PIF) schemes used in kinetic plasma simulations. The different strategies are i) domain decomposition, where both the particles and Fourier modes are split between the MPI ranks ii) particle decomposition, where only the particles are split between the ranks and each rank carries all the modes, and, iii) space-time decomposition, in which time parallelization based on the parareal algorithm is added on top of the particle decomposition. We describe the different communication patterns involved in each of the strategies, the parameter regimes where they work best, and explain their advantages and disadvantages. We implement the strategies within the open-source, performance portable library IPPL and conduct scaling studies with 3D-3V Landau damping and Penning trap benchmark problems on Alps and JUWELS booster supercomputers. We analyze the dominant component timings in each of the strategies and identify areas for future optimizations.

cs.CE↗

arXiv2026

A Comparison of Massively Parallel Performance Portable Particle-in-Cell schemes for electrostatic kinetic plasma simulations

We compare different Poisson solvers within the context of an electrostatic Vlasov-Poisson system. These schemes are implemented as part of the IPPL (Independent Parallel Particle Layer) library (Frey et al., 2024), which provides performance portable and dimension independent building blocks for scientific simulations requiring particle-mesh methods, with Eulerian (mesh-based) and Lagrangian (particle-based) approaches. The simulation used to compare the performance and portability of the schemes is Landau damping, part of a set of mini-applications implemented to benchmark and showcase the capabilities of the IPPL library (Muralikrishnan et al., 2024). We use grid-sizes of $512^3$ and $1024^3$ with 8 particles per cell, running with different algorithms in the solve phase of the Particle-in-Cell (PIC) loop: a Fast Fourier Transform (FFT) pseudo-spectral solver, a matrix-free finite difference Preconditioned Conjugate Gradient (PCG) solver, and a matrix-free Finite Element (FEM) solver. We also compare these PIC schemes to the novel Particle-in-Fourier (PIF) scheme, which performs interpolations using non-uniform FFTs thereby avoiding a grid in the real space. We obtain results on different computing architectures, such as AMD GPUs (LUMI at CSC), and Nvidia GPUs (Alps at CSCS and JUWELS Booster at Jülich Supercomputing Center), showcasing portability. In terms of absolute time the FFT solver is advantageous, but is limited in its applicability. All other field solvers in the PIC scheme are an order-of-magnitude more expensive in terms of time, but scale similarly to the FFT case in the electrostatic PIC context. The PIF scheme serves as a high fidelity alternative to standard PIC, and while it is costlier than the FFT-based PIC scheme, it shows excellent scalability on all the architectures.

cs.CE↗

arXiv2025

A Massively Parallel Performance Portable Free-space Spectral Poisson Solver

Vico et al. (2016) suggest a fast algorithm for computing volume potentials, beneficial to fields with problems requiring the solution of the free-space Poisson's equation, such as beam and plasma physics. Currently, the standard is the algorithm of Hockney and Eastwood (1988), with second order in convergence at best. The algorithm proposed by Vico et al. converges spectrally for sufficiently smooth functions i.e. faster than any fixed order in the number of grid points. We implement a performance portable version of the traditional Hockney-Eastwood and the novel Vico-Greengard Poisson solver as part of the IPPL (Independent Parallel Particle Layer) library. For sufficiently smooth source functions, the Vico-Greengard algorithm achieves higher accuracy than the Hockney-Eastwood method with the same grid size, reducing the computational demands of high resolution simulations since one could use coarser grids to achieve them. Additionally, we propose an improvement to the Vico-Greengard method which further reduces its memory footprint. This is important for GPUs, which have limited memory, and should be taken into account when selecting numerical algorithms for performance portable codes. Finally, we showcase performance through GPU and CPU scaling studies on the Perlmutter (NERSC) supercomputer, with efficiencies staying above 50% in the strong scaling case. To showcase portability, we also run the scaling studies on the Alps supercomputer at CSCS, Switzerland and the GPU partition of the Lumi supercomputer at CSC, Finland.

physics.comp-ph↗

arXiv2025

Error Analysis and Parallel Scaling Study of A Parareal Parallel-in-Time Integration Algorithm for Particle-in-Fourier Schemes

We propose a parareal based time parallelization scheme in the phase-space for the particle-in-Fourier (PIF) discretization of the Vlasov-Poisson system used in kinetic plasma simulations. We use PIF with a coarse tolerance for the nonuniform fast Fourier transforms, or the standard particle-in-cell scheme, combined with temporal coarsening, as coarse propagators. This is different from the typical spatial coarsening of particles and/or Fourier modes for parareal, which are not possible or effective for PIF schemes. We perform an error analysis of the algorithm and verify the results numerically with Landau damping, two-stream instability, and Penning trap test cases in 3D-3V. We also implement the space-time parallelization of the PIF schemes in the open-source, performance-portable library IPPL and conduct scaling studies up to 1536 A100 GPUs on the JUWELS booster supercomputer. The space-time parallelization utilizing the parareal algorithm for the time parallelization provides up to $4-6$ times speedup compared to spatial parallelization alone and achieves a push rate of around 1 billion particles per second for the benchmark plasma mini-apps considered.

math.NA↗

arXiv2024

A particle-in-Fourier method with semi-discrete energy conservation for non-periodic boundary conditions

We introduce a novel particle-in-Fourier (PIF) scheme that extends its applicability to non-periodic boundary conditions. Our method handles free space boundary conditions by replacing the Fourier Laplacian operator in PIF with a mollified Green's function as first introduced by Vico-Greengard-Ferrando. This modification yields highly accurate free space solutions to the Vlasov-Poisson system, while still maintaining energy conservation up to an error bounded by the time step size. We also explain how to extend our scheme to arbitrary Dirichlet boundary conditions via standard potential theory, which we illustrate in detail for Dirichlet boundary conditions on a circular boundary. We support our approach with proof-of-concept numerical results from two-dimensional plasma test cases to demonstrate the accuracy, efficiency, and conservation properties of the scheme. By avoiding grid heating and finite grid instability we are able to show an order of magnitude speedup compared to the standard PIC scheme for a long time integration cyclotron simulation.

math.NA↗

arXiv2022

Scaling and performance portability of the particle-in-cell scheme for plasma physics applications through mini-apps targeting exascale architectures

We perform a scaling and performance portability study of the particle-in-cell scheme for plasma physics applications through a set of mini-apps we name "Alpine", which can make use of exascale computing capabilities. The mini-apps are based on Independent Parallel Particle Layer, a framework that is designed around performance portable and dimension independent particles and fields. We benchmark the simulations with varying parameters such as grid resolutions ($512^3$ to $2048^3$) and number of simulation particles ($10^9$ to $10^{11}$) with the following mini-apps: weak and strong Landau damping, bump-on-tail and two-stream instabilities, and the dynamics of an electron bunch in a charge-neutral Penning trap. We show strong and weak scaling and analyze the performance of different components on several pre-exascale architectures such as Piz-Daint, Cori, Summit and Perlmutter. While the scaling and portability study helps identify the performance critical components of the particle-in-cell scheme in the current state-of-the-art computing architectures, the mini-apps by themselves can be used to develop new algorithms and optimize their high performance implementations targeting exascale architectures.

physics.comp-ph↗

arXiv2021

Order-of-Magnitude Beam Current Improvement in Compact Cyclotrons

There is great need for high intensity proton beams from compact particle accelerators in particle physics, medical isotope production, and materials- and energy-research. To address this need, we present, for the first time, a design for a compact isochronous cyclotron that will be able to deliver 10 mA of 60 MeV protons - an order of magnitude higher than on-market compact cyclotrons and a factor four higher than research machines. A key breakthrough is that vortex motion is incorporated in the design of a cyclotron, leading to clean extraction. Beam losses on the septa of the electrostatic extraction channels stay below 50 W (a factor four below the required safety limit), while maintaining good beam quality. We present a set of highly accurate particle-in-cell simulations, and an uncertainty quantification of select beam input parameters using machine learning, showing the robustness of the design. This design can be utilized for beams for experiments in particle and nuclear physics, materials science and medical physics as well as for industrial applications.

physics.acc-ph↗

arXiv2020

A Multilevel Block Preconditioner for the HDG Trace System Applied to Incompressible Resistive MHD

We present a scalable block preconditioning strategy for the trace system coming from the high-order hybridized discontinuous Galerkin (HDG) discretization of incompressible resistive magnetohydrodynamics (MHD). We construct the block preconditioner with a least squares commutator (BFBT) approximation for the inverse of the Schur complement that segregates out the pressure unknowns of the trace system. The remaining velocity, magnetic field, and Lagrange multiplier unknowns form a coupled nodal unknown block (the upper block), for which a system algebraic multigrid (AMG) is used for the approximate inverse. The complexity of the MHD equations together with the algebraic nature of the statically condensed HDG trace system makes the choice of smoother in the system AMG part critical for the convergence and performance of the block preconditioner. Our numerical experiments show GMRES preconditioned by ILU(0) of overlap zero as a smoother inside system AMG performs best in terms of robustness, time per nonlinear iteration and memory requirements. With several transient test cases in 2D and 3D including the island coalescence problem at high Lundquist number we demonstrate the robustness and parallel scalability of the block preconditioner. Additionally for the upper block a preliminary study of an alternate nodal block system solver based on a multilevel approximate nested dissection is presented. On a 2D island coalescence problem the multilevel approximate nested dissection preconditioner shows better scalability with respect to mesh refinement than the system AMG, but is relatively less robust with respect to Lundquist number scaling.

math.NA↗

arXiv2020

Sparse Grids based Adaptive Noise Reduction strategy for Particle-In-Cell schemes

We propose a sparse grids based adaptive noise reduction strategy for electrostatic particle-in-cell (PIC) simulations. Our approach is based on the key idea of relying on sparse grids instead of a regular grid in order to increase the number of particles per cell for the same total number of particles, as first introduced in Ricketson and Cerfon (Plasma Phys. and Control. Fusion, 59(2), 024002). Adopting a new filtering perspective for this idea, we construct the algorithm so that it can be easily integrated into high performance large-scale PIC code bases. Unlike the physical and Fourier domain filters typically used in PIC codes, our approach automatically adapts to mesh size, number of particles per cell, smoothness of the density profile and the initial sampling technique. Thanks to the truncated combination technique, we can reduce the larger grid-based error of the standard sparse grids approach for non-aligned and non-smooth functions. We propose a heuristic based on formal error analysis for selecting the optimal truncation parameter at each time step, and develop a natural framework to minimize the total error in sparse PIC simulations. We demonstrate its efficiency and performance by means of two test cases: the diocotron instability in two dimensions, and the three-dimensional electron dynamics in a Penning trap. Our run time performance studies indicate that our new scheme can provide significant speedup and memory reduction as compared to regular PIC for achieving comparable accuracy in the charge density deposition.

physics.comp-ph↗

arXiv2019

A Multilevel Approach for Trace System in HDG Discretizations

We propose a multilevel approach for trace systems resulting from hybridized discontinuous Galerkin (HDG) methods. The key is to blend ideas from nested dissection, domain decomposition, and high-order characteristic of HDG discretizations. Specifically, we first create a coarse solver by eliminating and/or limiting the front growth in nested dissection. This is accomplished by projecting the trace data into a sequence of same or high-order polynomials on a set of increasingly $h-$coarser edges/faces. We then combine the coarse solver with a block-Jacobi fine scale solver to form a two-level solver/preconditioner. Numerical experiments indicate that the performance of the resulting two-level solver/preconditioner depends only on the smoothness of the solution and is independent of the nature of the PDE under consideration. While the proposed algorithms are developed within the HDG framework, they are applicable to other hybrid(ized) high-order finite element methods. Moreover, we show that our multilevel algorithms can be interpreted as a multigrid method with specific intergrid transfer and smoothing operators. With several numerical examples from Poisson, pure transport, and convection-diffusion equations we demonstrate the robustness and scalability of the algorithms.

math.NA↗

arXiv2018

Unified geometric multigrid algorithm for hybridized high-order finite element methods

We consider a standard elliptic partial differential equation and propose a geometric multigrid algorithm based on Dirichlet-to-Neumann (DtN) maps for hybridized high-order finite element methods. The proposed unified approach is applicable to any locally conservative hybridized finite element method including multinumerics with different hybridized methods in different parts of the domain. For these methods, the linear system involves only the unknowns residing on the mesh skeleton, and constructing intergrid transfer operators is therefore not trivial. The key to our geometric multigrid algorithm is the physics-based energy-preserving intergrid transfer operators which depend only on the fine scale DtN maps. Thanks to these operators, we completely avoid upscaling of parameters and no information regarding subgrid physics is explicitly required on coarse meshes. Moreover, our algorithm is agglomeration-based and can straightforwardly handle unstructured meshes. We perform extensive numerical studies with hybridized mixed methods, hybridized discontinuous Galerkin method, weak Galerkin method, and a hybridized version of interior penalty discontinuous Galerkin methods on a range of elliptic problems including subsurface flow through highly heterogeneous porous media. We compare the performance of different smoothers and analyze the effect of stabilization parameters on the scalability of the multigrid algorithm.

math.NA↗

arXiv2017

An Improved Iterative HDG Approach for Partial Differential Equations

We propose and analyze an iterative high-order hybridized discontinuous Galerkin (iHDG) discretization for linear partial differential equations. We improve our previous work (SIAM J. Sci. Comput. Vol. 39, No. 5, pp. S782--S808) in several directions: 1) the improved iHDG approach converges in a finite number of iterations for the scalar transport equation; 2) it is unconditionally convergent for both the linearized shallow water system and the convection-diffusion equation; 3) it has improved stability and convergence rates; 4) we uncover a relationship between the number of iterations and time stepsize, solution order, meshsize and the equation parameters. This allows us to choose the time stepsize such that the number of iterations is approximately independent of the solution order and the meshsize; and 5) we provide both strong and weak scalings of the improved iHDG approach up to $16,384$ cores. A connection between iHDG and time integration methods such as parareal and implicit/explicit methods are discussed. Extensive numerical results are presented to verify the theoretical findings.

math.NA↗

arXiv2017

iHDG: An Iterative HDG Framework for Partial Differential Equations

We present a scalable iterative solver for high-order hybridized discontinuous Galerkin (HDG) discretizations of linear partial differential equations. It is an interplay between domain decomposition methods and HDG discretizations, and hence inheriting advances from both sides. In particular, the method can be viewed as a Gauss-Seidel approach that requires only independent element-by-element and face-by-face local solves in each iteration. As such, it is well-suited for current and future computing systems with massive concurrencies. Unlike conventional Gauss-Seidel schemes which are purely algebraic, the convergence of iHDG, thanks to the built-in HDG numerical flux, does not depend on the ordering of unknowns. We rigorously show the convergence of the proposed method for the transport equation, the linearized shallow water equation and the convection-diffusion equation. For the transport equation, the method is convergent regardless of mesh size $h$ and solution order $p$, and furthermore the convergence rate is independent of the solution order. For the linearized shallow water and the convection-diffusion equations we show that the convergence is conditional on both $h$ and $p$. Extensive steady and time-dependent numerical results for the 2D and 3D transport equations, the linearized shallow water equation, and the convection-diffusion equation are presented to verify the theoretical findings.

math.NA↗

arXiv2016

eHDG:An Exponentially Convergent Iterative Solver for HDG Discretizations of Hyperbolic Partial Differential Equations

We present a scalable and efficient iterative solver for high-order hybridized discontinuous Galerkin (HDG) discretizations of hyperbolic partial differential equations. It is an interplay between domain decomposition methods and HDG discretizations. In particular, the method is a fixed-point approach that requires only independent element-by-element local solves in each iteration. As such, it is well-suited for current and future computing systems with massive concurrencies. We rigorously show that the proposed method is exponentially convergent in the number of iterations for transport and linearized shallow water equations. Furthermore, the convergence is independent of the solution order. Various 2D and 3D numerical results for steady and time-dependent problems are presented to verify our theoretical findings.

math.NA↗