RAMSES-GPU: Cell-by-Cell Adaptive Mesh Refinement with Magneto-Hydrodynamics and Self-Gravity on Graphics Processing Units
We present the implementation and optimization of the cosmological simulation code RAMSES on Graphics Processing Units (GPUs) using CUDA Fortran. This accelerated version ports the main computational routines, including hydrodynamics, particle dynamics, and self-gravity, to multi-GPU architectures. We detail our strategy for managing cell-by-cell Adaptive Mesh Refinement (AMR) on the GPU, utilizing bucket sort with prefix sums for AMR level sorting, radix sort via the CUB library for Hilbert key ordering, and an fnv64 hash table with linear probing for fast spatial indexing. Portability across diverse hardware architectures is achieved via a dispatcher and C-Fortran wrappers, calling CUDA, HIP, and Metal kernels directly translated from the CUDA Fortran framework. Hydrodynamics updates are executed via a Godunov MUSCL-Hancock HLLC Riemann solver managed through a three-tier shared-memory kernel architecture (named rock, paper, and scissor). Particle mass deposition uses Cloud-in-Cell (CIC) interpolation optimized with atomic additions or prefix sums, combined with a kick-drift-kick time integration pusher. Self-gravity is handled via a Multigrid (MG) Poisson solver performing hierarchical V-cycles on individual levels. Performance benchmarks conducted on NVIDIA A100 and H200 GPUs demonstrate substantial accelerations compared to multi-core CPUs, yielding 10x up to a 100x speedup for standard test problems such as the Sedov blast wave, molecular core collapse, and cosmological simulations. Finally, we briefly discuss additional accelerated physics modules, including equilibrium cooling, polytropic equations of state, ideal and non-ideal magneto-hydrodynamics (MHD), and stellar feedback.