SearcharxivSearch

arXiv subjects

Eleni Mathioulaki

Publications and source records attributed to Eleni Mathioulaki.

2 recordsLinked to original sources

Unifying von-Neumann HPC and Neuromorphic Acceleration via the EBRAINS Research Infrastructure: A Framework for High-Performance Workflows

Modern scientific workflows increasingly span diverse computing architectures, yet executing a single computational model across disparate systems often forces researchers to maintain fragmented, site-specific pipelines. In this paper, we address this challenge within the domain of computational neuroscience by presenting a unified, cloud-based workflow orchestrated via EBRAINS JupyterLab. This workflow enables users to transparently execute spiking neural networks on both von-Neumann supercomputers and neuromorphic hardware. Using a single federated identity, the system dispatches jobs to HPC sites (JUSUF, Galileo100) via PyUNICORE and to the SpiNNaker-1 neuromorphic system via the Neuromorphic Computing Platform Interface. To guarantee cross-site reproducibility and mitigate software version drift, we utilize a zero-installation execution mode that dynamically pulls PMIx-aware Apptainer containers to HPC compute nodes. Furthermore, we demonstrate genuine model-level portability using the NESTML domain-specific language, allowing custom neuron models to be written once and automatically compiled for either the NEST (C++) or sPyNNaker backends. Validated with a balanced random network case study, this work illustrates a practical, end-to-end path for hardware-agnostic workflows while highlighting the critical role of containerization and domain-specific languages in achieving true cross-platform reproducibility.

cs.DC

HPC Containers for EBRAINS: Towards Portable Cross-Domain Software Environment

Deploying complex, distributed scientific workflows across diverse HPC sites is often hindered by site-specific dependencies and complex build environments. This paper investigates the design and performance of portable HPC container images capable of encapsulating MPI- and CUDA-enabled software stacks without sacrificing bare-metal performance. This work is part of recent work performed within the EBRAINS Research Infrastructure, to evaluate the implementation of portable HPC (Apptainer-based) container images targeting the EBRAINS Software Distribution (ESD) -- a Spack-based software ecosystem comprising approximately 80 top-level packages (and 800 dependencies). We evaluate a hybrid, PMIx-based containerization strategy using Apptainer that seamlessly bypasses the need for site-specific builds by dynamically leveraging host-level specialized hardware, such as network interfaces and GPUs, on two production HPC clusters: Karolina and Jureca-DC. We demonstrate the feasibility of building portable, MPI- and CUDA-enabled scientific software into container images that correctly leverage site-installed drivers and hardware to reproduce bare-metal communication behavior. Using communication microbenchmarks (e.g., OSU and NCCL) alongside performance metrics of applications from neuroscience, we measure and verify their performance against bare-metal deployments. Crucially, our verification approach extends beyond top-level runtime measurements; we highlight the analysis of underlying debug logs to actively detect misbehavior and misconfigurations, such as suboptimal transport pathways. Ultimately, this investigation demonstrates the feasibility of a simple and reproducible methodology for decoupling software environments from underlying infrastructures, paving the way for automated pipelines that ensure optimized, performance-verified execution across varied HPC architectures.

cs.DC