arXiv subjects

Markus Rampp

Publications and source records attributed to Markus Rampp.

30 records · Page 2Linked to original sources

arXiv2019

Evaluation of performance portability frameworks for the implementation of a particle-in-cell code

This paper reports on an in-depth evaluation of the performance portability frameworks Kokkos and RAJA with respect to their suitability for the implementation of complex particle-in-cell (PIC) simulation codes, extending previous studies based on codes from other domains. At the example of a particle-in-cell model, we implemented the hotspot of the code in C++ and parallelized it using OpenMP, OpenACC, CUDA, Kokkos, and RAJA, targeting multi-core (CPU) and graphics (GPU) processors. Both, Kokkos and RAJA appear mature, are usable for complex codes, and keep their promise to provide performance portability across different architectures. Comparing the obtainable performance on state-of-the art hardware, but also considering aspects such as code complexity, feature availability, and overall productivity, we finally draw the conclusion that the Kokkos framework would be suited best to tackle the massively parallel implementation of the full PIC model.

cs.DC↗

arXiv2019

Cross-validation tests for cryo-EM maps using an independent particle set

Cryo-electron microscopy is a revolutionary technique that can provide 3D density maps at near-atomic resolution. However, map validation is still an open issue in the field. Despite several efforts from the community, it is possible to overfit the reconstructions to noisy data. Here, inspired by modern statistics, we develop a novel methodology that uses a small independent particle set to validate the 3D maps. The main idea is to monitor how the map probability evolves over the control set during the refinement. The method is complementary to the gold-standard procedure, which generates two reconstructions at each iteration. We low-pass filter the two reconstructions for different frequency cutoffs, and we calculate the probability of each filtered map given the control set. For high-quality maps, the probability should increase as a function of the frequency cutoff and of the refinement iteration. We also compute the similarity between the probability distributions of the two reconstructions. As higher frequencies are added to the maps, more dissimilar are the distributions. We optimized the BioEM software package to perform these calculations, and tested the method on several systems, some which were overfitted. Our results show that our method is able to discriminate the overfitted sets from the non-overfitted ones. We conclude that having a control particle set, not used for the refinement, is essential for cross-validating cryo-EM maps.

physics.bio-ph↗

arXiv2019

A massively parallel semi-Lagrangian solver for the six-dimensional Vlasov-Poisson equation

This paper presents an optimized and scalable semi-Lagrangian solver for the Vlasov-Poisson system in six-dimensional phase space. Grid-based solvers of the Vlasov equation are known to give accurate results. At the same time, these solvers are challenged by the curse of dimensionality resulting in very high memory requirements, and moreover, requiring highly efficient parallelization schemes. In this paper, we consider the 6d Vlasov-Poisson problem discretized by a split-step semi-Lagrangian scheme, using successive 1d interpolations on 1d stripes of the 6d domain. Two parallelization paradigms are compared, a remapping scheme and a classical domain decomposition approach applied to the full 6d problem. From numerical experiments, the latter approach is found to be superior in the massively parallel case in various respects. We address the challenge of artificial time step restrictions due to the decomposition of the domain by introducing a blocked one-sided communication scheme for the purely electrostatic case and a rotating mesh for the case with a constant magnetic field. In addition, we propose a pipelining scheme that enables to hide the costs for the halo communication between neighbor processes efficiently behind useful computation. Parallel scalability on up to 65k processes is demonstrated for benchmark problems on a supercomputer.

physics.comp-ph↗

arXiv2017

Hydrodynamic turbulence in quasi-Keplerian rotating flows

We report a direct-numerical-simulation study of Taylor-Couette flow in the quasi-Keplerian regime at shear Reynolds numbers up to $\mathcal{O}(10^5)$. Quasi-Keplerian rotating flow has been investigated for decades as a simplified model system to study the origin of turbulence in accretion disks that is not fully understood. The flow in this study is axially periodic and thus the experimental end-wall effects on the stability of the flow are avoided. Using optimal linear perturbations as initial conditions, our simulations find no sustained turbulence: the strong initial perturbations distort the velocity profile and trigger turbulence that eventually decays.

physics.flu-dyn↗

arXiv2016

BioEM: GPU-accelerated computing of Bayesian inference of electron microscopy images

In cryo-electron microscopy (EM), molecular structures are determined from large numbers of projection images of individual particles. To harness the full power of this single-molecule information, we use the Bayesian inference of EM (BioEM) formalism. By ranking structural models using posterior probabilities calculated for individual images, BioEM in principle addresses the challenge of working with highly dynamic or heterogeneous systems not easily handled in traditional EM reconstruction. However, the calculation of these posteriors for large numbers of particles and models is computationally demanding. Here we present highly parallelized, GPU-accelerated computer software that performs this task efficiently. Our flexible formulation employs CUDA, OpenMP, and MPI parallelization combined with both CPU and GPU computing. The resulting BioEM software scales nearly ideally both on pure CPU and on CPU+GPU architectures, thus enabling Bayesian analysis of tens of thousands of images in a reasonable time. The general mathematical framework and robust algorithms are not limited to cryo-electron microscopy but can be generalized for electron tomography and other imaging experiments.

q-bio.BM↗

arXiv2016

GPEC, a real-time capable Tokamak equilibrium code

A new parallel equilibrium reconstruction code for tokamak plasmas is presented. GPEC allows to compute equilibrium flux distributions sufficiently accurate to derive parameters for plasma control within 1 ms of runtime which enables real-time applications at the ASDEX Upgrade experiment (AUG) and other machines with a control cycle of at least this size. The underlying algorithms are based on the well-established offline-analysis code CLISTE, following the classical concept of iteratively solving the Grad-Shafranov equation and feeding in diagnostic signals from the experiment. The new code adopts a hybrid parallelization scheme for computing the equilibrium flux distribution and extends the fast, shared-memory-parallel Poisson solver which we have described previously by a distributed computation of the individual Poisson problems corresponding to different basis functions. The code is based entirely on open-source software components and runs on standard server hardware and software environments. The real-time capability of GPEC is demonstrated by performing an offline-computation of a sequence of 1000 flux distributions which are taken from one second of operation of a typical AUG discharge and deriving the relevant control parameters with a time resolution of a millisecond. On current server hardware the new code allows employing a grid size of 32x64 zones for the spatial discretization and up to 15 basis functions. It takes into account about 90 diagnostic signals while using up to 4 equilibrium iterations and computing more than 20 plasma-control parameters, including the computationally expensive safety-factor q on at least 4 different levels of the normalized flux.

physics.plasm-ph↗

arXiv2014

A Hybrid MPI-OpenMP Parallel Implementation for pseudospectral simulations with application to Taylor-Couette Flow

A hybrid-parallel direct-numerical-simulation method with application to turbulent Taylor-Couette flow is presented. The Navier-Stokes equations are discretized in cylindrical coordinates with the spectral Fourier-Galerkin method in the axial and azimuthal directions, and high-order finite differences in the radial direction. Time is advanced by a second-order, semi-implicit projection scheme, which requires the solution of five Helmholtz/Poisson equations, avoids staggered grids and renders very small slip velocities. Nonlinear terms are computed with the pseudospectral method. The code is parallelized using a hybrid MPI-OpenMP strategy, which is simpler to implement, reduces inter-node communications and is more efficient compared to a flat MPI parallelization. A strong scaling study shows that the hybrid code maintains very good scalability up to more than 20000 processor cores and thus allows to perform simulations at higher resolutions than previously feasible, and opens up the possibility to simulate turbulent Taylor-Couette flows at Reynolds numbers up to $\mathcal{O}(10^5)$. This enables to probe hydrodynamic turbulence in Keplerian flows in experimentally relevant regimes.

physics.comp-ph↗

arXiv2014

Towards Petaflops Capability of the VERTEX Supernova Code

The VERTEX code is employed for multi-dimensional neutrino-radiation hydrodynamics simulations of core-collapse supernova explosions from first principles. The code is considered state-of-the-art in supernova research and it has been used for modeling for more than a decade, resulting in numerous scientific publications. The computational performance of the code, which is currently deployed on several high-performance computing (HPC) systems up to the Tier-0 class (e.g. in the framework of the European PRACE initiative and the German GAUSS program), however, has so far not been extensively documented. This paper presents a high-level overview of the relevant algorithms and parallelization strategies and outlines the technical challenges and achievements encountered along the evolution of the code from the gigaflops scale with the first, serial simulations in 2000, up to almost petaflops capabilities, as demonstrated lately on the SuperMUC system of the Leibniz Supercomputing Centre (LRZ). In particular, we shall document the parallel scalability and computational efficiency of VERTEX at the large scale and on the major, contemporary HPC platforms. We will outline upcoming scientific requirements and discuss the resulting challenges for the future development and operation of the code.

physics.comp-ph↗

arXiv2013

Porting Large HPC Applications to GPU Clusters: The Codes GENE and VERTEX

We have developed GPU versions for two major high-performance-computing (HPC) applications originating from two different scientific domains. GENE is a plasma microturbulence code which is employed for simulations of nuclear fusion plasmas. VERTEX is a neutrino-radiation hydrodynamics code for "first principles"-simulations of core-collapse supernova explosions. The codes are considered state of the art in their respective scientific domains, both concerning their scientific scope and functionality as well as the achievable compute performance, in particular parallel scalability on all relevant HPC platforms. GENE and VERTEX were ported by us to HPC cluster architectures with two NVidia Kepler GPUs mounted in each node in addition to two Intel Xeon CPUs of the Sandy Bridge family. On such platforms we achieve up to twofold gains in the overall application performance in the sense of a reduction of the time to solution for a given setup with respect to a pure CPU cluster. The paper describes our basic porting strategies and benchmarking methodology, and details the main algorithmic and technical challenges we faced on the new, heterogeneous architecture.

physics.comp-ph↗

arXiv2003

Towards Gravitational Wave Signals from Realistic Core Collapse Supernova Models

We have computed the gravitational wave signal from supernova core collapse using the presently most realistic input physics available. We start from state-of-the-art progenitor models of rotating and non-rotating massive stars, and simulate the dynamics of their core collapse by integrating the equations of axisymmetric hydrodynamics together with the Boltzmann equation for the neutrino transport including an elaborate description of neutrino interactions, and a realistic equation of state. We compute the quadrupole wave amplitudes, the Fourier wave spectra, the amount of energy radiated in form of gravitational waves, and the S/N ratios for the LIGO and the tuned Advanced LIGO interferometers resulting both from non-radial mass motion and anisotropic neutrino emission. The simulations demonstrate that the dominant contribution to the gravitational wave signal is produced by neutrino-driven convection behind the supernova shock. For stellar cores rotating at the extreme of current stellar evolution predictions, the core-bounce signal is detectable with advanced LIGO up to a distance of 5kpc, whereas the signal from post-shock convection is observable up to a distance of about 100kpc. If the core is non-rotating its gravitational wave emission can be measured up to a distance of 15kpc, while the signal from the Ledoux convection in the deleptonizing, nascent neutron star can be detected up to a distance of 10kpc. Both kinds of signals are generically produced by convection in any core collapse supernova.

astro-ph↗

arXiv2002

Radiation hydrodynamics with neutrinos: Variable Eddington factor method for core-collapse supernova simulations

Neutrino transport and neutrino interactions in dense matter play a crucial role in stellar core collapse, supernova explosions and neutron star formation. Here we present a detailed description of a new numerical code for treating the time and energy dependent neutrino transport in hydrodynamical simulations of such events. The code is based on a variable Eddington factor method to deal with the integro-differential character of the Boltzmann equation. The moments of the neutrino distribution function and the energy and lepton number exchange with the stellar medium are determined by iteratively solving the zeroth and first order moment equations in combination with a model Boltzmann equation. The latter is discretized on a grid of tangent rays. The integration of the transport equations and the neutrino source terms is performed in a time-implicit way. In the present version of the program, the transport part is coupled to an explicit hydrodynamics code which follows the evolution of the stellar plasma by a finite-volume method with piecewise parabolic interpolation, using a Riemann solver for calculating the hydrodynamic states. The neutrino source terms are implemented in an operator-split step. Neutrino transport and hydrodynamics can be calculated with different spatial grids and different time steppings. The structure of the described code is modular and offers a high degree of flexibility for an application to relativistic and multi-dimensional problems at different levels of refinement and accuracy. We critically evaluate results for a number of test cases, including neutrino transport in rapidly moving stellar media and approximate relativistic core collapse, and suggest a path for generalizing the code to be used in multi-dimensional simulations of convection in neutron stars and supernovae.

astro-ph↗

arXiv2002

Electron Neutrino Pair Annihilation: A New Source for Muon and Tau Neutrinos in Supernovae

We show that in a supernova core the annihilation process nu_e nu_e-bar -> nu_{mu,tau} nu_{mu,tau}-bar is always more important than the traditional reaction e^+ e^- -> nu_{mu,tau} nu_{mu,tau}-bar as a source for muon and tau neutrino pairs. We study the impact of the new process by means of a Monte Carlo transport code with a static stellar background model and by means of a self-consistent hydrodynamical simulation with Boltzmann neutrino transport. Nucleon bremsstrahlung NN -> NN nu_{mu,tau} nu_{mu,tau}-bar is also included as another important source term. Taking into account nu_e nu_e-bar -> nu_{mu,tau} nu_{mu,tau}-bar increases the nu_mu and nu_tau luminosities by as much as 20% while the spectra remain almost unaffected. In our hydrodynamical simulation the shock was somewhat weakened. Elastic nu_{mu,tau} nu_e and nu_{mu,tau} nu_e scattering is not negligible but less important than nu_{mu,tau} e^+ or e^- scattering. Its influence on the nu_{mu,tau} fluxes and spectra is small after all other processes have been included.

astro-ph↗