arXiv subjects

Mladen Ivkovic

Publications and source records attributed to Mladen Ivkovic.

7 recordsLinked to original sources

arXiv2026

Memory Layouts for GPU-Data Transfer Buffering in SPH

The rise in GPU compute speed has outpaced improvements in host-to-device memory transfer speeds, despite the advent of shared-memory superchips. Consequently, memory transfer times now constitute an increasingly large fraction of total time-to-solution, compelling developers to compress GPU kernel input and output data into compact, minimal formats prior to GPU-offloading. This complements existing work on GPU- and compute-friendly data arrangements. We study a Smoothed Particle Hydrodynamics solver and propose memory layout strategies for host-side particle data that are particularly well-suited to GPU-offloading. Specifically, we advocate splitting classic array-of-struct data structures into a split array-of-struct arrangement, in which each logical struct decomposes into substructs determined by kernel read/write access patterns and attribute types. Splitting a monolithic particle struct into several bespoke, finer-grained structs can reduce the time required to pack data to and from buffers by ~20% - 40%, lowering total time spent on GPU-offloading by ~12% - 25%.

cs.DC↗

arXiv2026

SPARCS -- combining radiation hydrodynamics with non-equilibrium metal chemistry in the SWIFT astrophysical code

We present SPARCS, which combines the moment-based radiative transfer method SPH-M1RT with the non-equilibrium metal chemistry solver CHIMES in the modern highly-parallel astrophysical code SWIFT. SPARCS enables on-the-fly radiation hydrodynamics simulations, with multi-frequency ultraviolet radiative transfer coupled with all ionisation states of 11 major elements, in the presence of dust, cosmic ray ionization and heating, and self-gravity. Direct radiation pressure on gas and dust is also accounted for. We validate SPARCS against analytic solutions and standard photo-ionization codes such as CLOUDY in idealized tests. As an example application, we simulate an ionization front propagating through an inhomogeneous interstellar medium with solar metallicity. We produce mock optical emission line observations with the level population calculation code HyLight and the diagnostic radiative transfer code RADMC3D. We find that non-equilibrium effects and inhomogeneities can boost the low ion fractions by up to an order of magnitude. Possible applications of SPARCS include studying the dynamical impact of radiation on gas in star-forming regions, and in the interstellar and circumgalactic medium, as well as interpreting line diagnostics in such environments, and galactic or AGN outflows.

astro-ph.GA↗

arXiv2026

Task-parallelism in SWIFT for heterogeneous compute architectures

This paper highlights first steps towards enabling graphics processing unit (GPU) acceleration of the task-parallel smoothed particle hydrodynamics (SPH) solver SWIFT. Novel combinations of algorithms are presented, enabling SWIFT to function as a truly heterogeneous software leveraging task-parallelism on CPUs for memory-bound computations concurrently with GPUs for compute-bound computations while minimising the effects of CPU-GPU communication latency. The proposed algorithms are validated in extensive testing. The GPU acceleration methodology is shown to deliver up to 3.5 and 7.5 speedups for the offloaded computations when including and excluding the time required to prepare and post-process data transfers on the CPU side, respectively. The overall performance of the GPU-accelerated hydrodynamic solver for a full simulation on a single Grace-Hopper superchip is 1.8 times faster compared to the superchips fully parallelised CPU capabilities. This constitutes an improvement from 8 million particle updates/s for the full CPU-only baseline (115,000 updates per CPU core) to 15 million updates/s for the GPU-accelerated SPH solver. Moreover, it displays near-perfect strong scaling on 4 Grace-Hopper nodes. The GPU-acceleration is also demonstrated to give a 29 percent improvement in energy efficiency in comparison to CPU-only baselines. Finally, inter-influential bottlenecks in the prototype solver presented in this work are identified: A significant amount of time (up to 80 percent) of a GPU-offloading cycle is spent on preparing and post-processing particle data on the CPU for the transfer to and from the GPU, respectively. Approaches are suggested to minimise their effects and maximise the solver's performance in our future work.

cs.PF↗

arXiv2025

An extension of C++ with memory-centric specifications for HPC to reduce memory footprints and streamline MPI development

C++ leans towards a memory-inefficient storage of structs: The compiler inserts padding bits, while it is not able to exploit knowledge about the range of integers, enums or bitsets. Furthermore, the language provides no support for arbitrary floating-point precisions. We propose a language extension based upon attributes through which developers can guide the compiler what memory arrangements would be beneficial: Can multiple booleans or integers with limited range be squeezed into one bit field, do floating-point numbers hold fewer significant bits than in the IEEE standard, and is a programmer willing to trade attribute ordering guarantees for a more compact object representation? The extension offers the opportunity to fall back to normal alignment and native C++ floating point representations via plain C++ assignments, no dependencies upon external libraries are introduced, and the resulting code remains (syntactically) standard C++. As MPI remains the de-facto standard for distributed memory calculations in C++, we furthermore propose additional attributes which streamline the MPI datatype modelling in combination with our memory optimisation extensions. Our work implements the language annotations within LLVM and demonstrates their potential impact through smoothed particle hydrodynamics benchmarks. They uncover the potential gains in terms of performance and development productivity.

cs.MS↗

arXiv2024

SWIFT: A modern highly-parallel gravity and smoothed particle hydrodynamics solver for astrophysical and cosmological applications

Numerical simulations have become one of the key tools used by theorists in all the fields of astrophysics and cosmology. The development of modern tools that target the largest existing computing systems and exploit state-of-the-art numerical methods and algorithms is thus crucial. In this paper, we introduce the fully open-source highly-parallel, versatile, and modular coupled hydrodynamics, gravity, cosmology, and galaxy-formation code SWIFT. The software package exploits hybrid shared- and distributed-memory task-based parallelism, asynchronous communications, and domain-decomposition algorithms based on balancing the workload, rather than the data, to efficiently exploit modern high-performance computing cluster architectures. Gravity is solved for using a fast-multipole-method, optionally coupled to a particle mesh solver in Fourier space to handle periodic volumes. For gas evolution, multiple modern flavours of Smoothed Particle Hydrodynamics are implemented. SWIFT also evolves neutrinos using a state-of-the-art particle-based method. Two complementary networks of sub-grid models for galaxy formation as well as extensions to simulate planetary physics are also released as part of the code. An extensive set of output options, including snapshots, light-cones, power spectra, and a coupling to structure finders are also included. We describe the overall code architecture, summarise the consistency and accuracy tests that were performed, and demonstrate the excellent weak-scaling performance of the code using a representative cosmological hydrodynamical problem with $\approx$$300$ billion particles. The code is released to the community alongside extensive documentation for both users and developers, a large selection of example test problems, and a suite of tools to aid in the analysis of large simulations run with SWIFT.

astro-ph.IM↗

arXiv2023

GEAR-RT: Towards Exa-Scale Moment Based Radiative Transfer For Cosmological Simulations Using Task-Based Parallelism And Dynamic Sub-Cycling with SWIFT

The development and implementation of GEAR-RT, a radiative transfer solver using the M1 closure in the open source code SWIFT, is presented, and validated using standard tests for radiative transfer. GEAR-RT is modeled after RAMSES-RT (Rosdahl et al. 2013) with some key differences. Firstly, while RAMSES-RT uses Finite Volume methods and an Adaptive Mesh Refinement (AMR) strategy, GEAR-RT employs particles as discretization elements and solves the equations using a Finite Volume Particle Method (FVPM). Secondly, GEAR-RT makes use of the task-based parallelization strategy of SWIFT, which allows for optimized load balancing, increased cache efficiency, asynchronous communications, and a domain decomposition based on work rather than on data. GEAR-RT is able to perform sub-cycles of radiative transfer steps w.r.t. a single hydrodynamics step. Radiation requires much smaller time step sizes than hydrodynamics, and sub-cycling permits calculations which are not strictly necessary to be skipped. Indeed, in a test case with gravity, hydrodynamics, and radiative transfer, the sub-cycling is able to reduce the runtime of a simulation by over 90%. Allowing only a part of the involved physics to be sub-cycled is a contrived matter when task-based parallelism is involved, and is an entirely novel feature in SWIFT. Since GEAR-RT uses a FVPM, a detailed introduction into Finite Volume methods and Finite Volume Particle Methods is presented. In astrophysical literature, two FVPM methods are written about: Hopkins (2015) have implemented one in their GIZMO code, while the one mentioned in Ivanova et al. (2013) isn't used to date. In this work, I test an implementation of the Ivanova et al. (2013) version, and conclude that in its current form, it is not suitable for use with particles which are co-moving with the fluid, which in turn is an essential feature for cosmological simulations.

astro-ph.IM↗

arXiv2021

ACACIA: a new method to produce on-the-fly merger trees in the RAMSES code

The implementation of ACACIA, a new algorithm to generate dark matter halo merger trees with the Adaptive Mesh Refinement (AMR) code RAMSES, is presented. The algorithm is fully parallel and based on the Message Passing Interface (MPI). As opposed to most available merger tree tools, it works on the fly during the course of the N body simulation. It can track dark matter substructures individually using the index of the most bound particle in the clump. Once a halo (or a sub-halo) merges into another one, the algorithm still tracks it through the last identified most bound particle in the clump, allowing to check at later snapshots whether the merging event was definitive, or whether it was only temporary, with the clump only traversing another one. The same technique can be used to track orphan galaxies that are not assigned to a parent clump anymore because the clump dissolved due to numerical over-merging. We study in detail the impact of various parameters on the resulting halo catalogues and corresponding merger histories. We then compare the performance of our method using standard validation diagnostics, demonstrating that we reach a quality similar to the best available and commonly used merger tree tools. As a proof of concept, we use our merger tree algorithm together with a parametrised stellar-mass-to-halo-mass relation and generate a mock galaxy catalogue that shows good agreement with observational data.

astro-ph.GA↗