arXiv subjects

Klaus Weide

Publications and source records attributed to Klaus Weide.

9 recordsLinked to original sources

arXiv2026

thornado+Flash-X: A Hybrid DG-IMEX and Finite-Volume Framework for Neutrino-Radiation Hydrodynamics in Core-Collapse Supernovae

We present neutrino-transport algorithms implemented in the toolkit for high-order neutrino-radiation hydrodynamics (thornado) and their coupling to self-gravitating hydrodynamics within the adaptive mesh refinement (AMR)-based multiphysics simulation framework Flash-X. thornado, developed primarily for simulations of core-collapse supernovae (CCSNe), employs a spectral, six-species two-moment formulation with algebraic closure and special-relativistic observer corrections accurate to $O(v/c)$, and uses discontinuous Galerkin (DG) methods for phase-space discretization combined with implicit-explicit time stepping. A key development is a nonlinear neutrino-matter coupling algorithm based on nested fixed-point iteration with Anderson acceleration, enabling fully implicit treatment of collisional processes, including energy-coupling interactions such as neutrino-electron scattering and pair production. Coupling to finite-volume (FV) hydrodynamics is achieved with a hybrid DG-FV representation of the fluid variables and operator-split evolution in Flash-X. The implementation is verified using basic transport tests with idealized opacities and relaxation and deleptonization problems with tabulated microphysics. Spherically symmetric CCSN simulations demonstrate accuracy and robustness of the coupled scheme, including close agreement with the CCSN simulation code Chimera. An axisymmetric CCSN simulation further demonstrates the viability of DG-based neutrino transport for multidimensional supernova modeling within Flash-X. thornado's neutrino-transport solver is GPU-enabled using OpenMP offloading or OpenACC, and all CCSN applications included in this work use the GPU implementation. Together, these results establish a foundation for future enhancements in physics fidelity, numerical algorithms, and computational performance, for increasingly realistic large-scale CCSN simulations.

astro-ph.IM↗

arXiv2025

ORCHA: A Performance Portability System for Extreme Heterogeneity

Heterogeneity is the prevalent trend in the rapidly evolving high-performance computing (HPC) landscape in both hardware and application software. The diversity in hardware platforms, currently comprising various accelerators and a future possibility of specializable chiplets, poses a significant challenge for scientific software developers aiming to harness optimal performance across different computing platforms while maintaining the quality of solutions when their applications are simultaneously growing more complex. Code synthesis and code generation can provide mechanisms to mitigate this challenge. We have developed a divide-and-conquer approach where different aspects of performance are handled by different stand-alone tools that are interfaced with the application through generated code. This system, ORCHA, enables users to configure and orchestrate their computations among available resources on a platform by specifying a high-level recipe, thereby permitting a many-to-many paradigm where each recipe results in a different variant of the application. The core design goal is to let users decide the application's hardware mapping and orchestration by editing only the high-level recipe. Tools in ORCHA distribution are: CG-Kit for translating the recipe into an execution graph; Milhoja to execute the graph by orchestrating data and task among hardware resources; and Macroprocessor that enables users to define their own code-shorthand for higher composability and easier management of code variants. In this paper, we describe the design of ORCHA and the role that code-generation plays in isolating applications from tools. We demonstrate the breadth of configurations ORCHA enables with a case study in which an application configuration is realized on three distinct hardware mappings---a GPU-centric, a CPU/GPU balanced, and a CPU/GPU concurrent layouts by using different recipes.

math.NA↗

arXiv2024

CG-Kit: Code Generation Toolkit for Performant and Maintainable Variants of Source Code Applied to Flash-X Hydrodynamics Simulations

CG-Kit is a new code generation toolkit that we propose as a solution for portability and maintainability for scientific computing applications. The development of CG-Kit is rooted in the urgent need created by the shifting landscape of high-performance computing platforms and the algorithmic complexities of a particular large-scale multiphysics application: Flash-X. This combination leads to unique challenges including handling an existing large code base in Fortran and/or C/C++, subdivision of code into a great variety of units supporting a wide range of physics and numerical methods, different parallelization techniques for distributed- and shared-memory systems and accelerator devices, and heterogeneity of computing platforms requiring coexisting variants of parallel algorithms. The challenges demand that developers determine custom abstractions and granularity for code generation. CG-Kit tackles this with standalone tools that can be combined into highly specific and, we argue, highly effective portability and maintainability tool chains. Here we present the design of our new tools: parametrized source trees, control flow graphs, and recipes. The tools are implemented in Python. Although the tools are agnostic to the programming language of the source code, we focus on C/C++ and Fortran. Code generation experiments demonstrate the generation of variants of parallel algorithms: first, multithreaded variants of the basic AXPY operation (scalar-vector addition and vector-vector multiplication) to introduce the application of CG-Kit tool chains; and second, variants of parallel algorithms within a hydrodynamics solver, called Spark, from Flash-X that operates on block-structured adaptive meshes. In summary, code generated by CG-Kit achieves a reduction by over 60% of the original C/C++/Fortran source code.

cs.DC↗

arXiv2023

Framework and Methodology for Verification of a Complex Scientific Simulation Software, Flash-X

Computational science relies on scientific software as its primary instrument for scientific discovery. Therefore, similar to the use of other types of scientific instruments, correct software and the correct operation of the software is necessary for executing rigorous scientific investigations. Scientific software verification can be especially difficult, as users typically need to modify the software as part of a scientific study. Systematic methodologies for building test suites for scientific software are rare in the literature. Here, we describe a methodology that we have developed for Flash-X, a community simulation software for multiple scientific domains, that has composable components that can be permuted and combined in a multitude of ways to generate a wide range of applications. Ensuring sufficient code coverage by a test suite is particularly challenging due to this composability. Our methodology includes a consideration of trade-offs between meeting software quality goals, developer productivity, and meeting the scientific goals of the Flash-X user community.

cs.SE↗

arXiv2022

Flash-X, a multiphysics simulation software instrument

Flash-X is a highly composable multiphysics software system that can be used to simulate physical phenomena in several scientific domains. It derives some of its solvers from FLASH, which was first released in 2000. Flash-X has a new framework that relies on abstractions and asynchronous communications for performance portability across a range of increasingly heterogeneous hardware platforms. Flash-X is meant primarily for solving Eulerian formulations of applications with compressible and/or incompressible reactive flows. It also has a built-in, versatile Lagrangian framework that can be used in many different ways, including implementing tracers, particle-in-cell simulations, and immersed boundary methods.

physics.comp-ph↗

arXiv2021

Checkpoint/Restart for Lagrangian particle mesh with AMR in community code FLASH-X

In this work we present the design decisions and advantages for accomplishing cross mesh format checkpoint-restart in community code FLASH-X. AMReX and Paramesh are the two AMR mesh formats developed and supported by FLASH-X. We also highlight strong and weak scaling study of existing HDF5 I/O checkpoint writing along with new ideas and results (presented during talk) for utilizing heterogeneous compute architectures for improved I/O performance.

cs.DC↗

arXiv2019

Gray Radiation Hydrodynamics with the FLASH Code for Astrophysical Applications

We present the newly-incorporated gray radiation hydrodynamics capabilities of the FLASH code based on a radiation flux-limiter aware hydrodynamics numerical implementation designed specifically for applications in astrophysical problems. The newly incorporated numerical methods consist of changes in the unsplit hydrodynamics solver and adjustments in the flux-limited radiation diffusion unit. Our method can treat problems in both the strong and weak radiation-matter coupling limits as well as transitions between the two regimes. Appropriate extensions in the "Helmholtz" equation of state are implemented to treat two-temperature astrophysical plasmas involving the interaction between radiation and matter and the addition of a new opacity unit based on the OPAL opacity database, commonly used for astrophysical fluids. A set of radiation-hydrodynamics test problems is presented aiming to showcase the new capabilities of FLASH and to provide direct comparison to other codes like CASTRO. To illustrate the capacity of FLASH to simulate phenomena occurring in stellar explosions, such as shock break-out, radiative precursors and supernova ejecta heating due to the decays of radioactive nickel-56 and cobalt-56, we also present 1D supernova simulations and compare the computed lightcurves to those of the SNEC code. The latest public release of FLASH with these enhanced capabilities is available for download and use by the broader astrophysics community.

astro-ph.SR↗

arXiv2016

A Survey of High Level Frameworks in Block-Structured Adaptive Mesh Refinement Packages

Over the last decade block-structured adaptive mesh refinement (SAMR) has found increasing use in large, publicly available codes and frameworks. SAMR frameworks have evolved along different paths. Some have stayed focused on specific domain areas, others have pursued a more general functionality, providing the building blocks for a larger variety of applications. In this survey paper we examine a representative set of SAMR packages and SAMR-based codes that have been in existence for half a decade or more, have a reasonably sized and active user base outside of their home institutions, and are publicly available. The set consists of a mix of SAMR packages and application codes that cover a broad range of scientific domains. We look at their high-level frameworks, and their approach to dealing with the advent of radical changes in hardware architecture. The codes included in this survey are BoxLib, Cactus, Chombo, Enzo, FLASH, and Uintah.

cs.DC↗

arXiv2015

The Biermann Catastrophe in Numerical MHD

The Biermann Battery effect is frequently invoked in cosmic magnetogenesis and studied in High-Energy Density laboratory physics experiments. Generation of magnetic fields by the Biermann effect due to mis-aligned density and temperature gradients in smooth flow behind shocks is well known. We show that a Biermann-effect magnetic field is also generated within shocks. Direct implementation of the Biermann effect in MHD codes does not capture this physical process, and worse, produces unphysical magnetic fields at shocks whose value does not converge with resolution. We show that this convergence breakdown is due to naive discretization, which fails to account for the fact that discretized irrotational vector fields have spurious solenoidal components that grow without bound near a discontinuity. We show that careful consideration of the kinetics of ion viscous shocks leads to a formulation of the Biermann effect that gives rise to a convergent algorithm. We note two novel physical effects: a resistive magnetic precursor in which Biermann-generated field in the shock "leaks" resistively upstream; and a thermal magnetic precursor , in which field is generated by the Biermann effect ahead of the shock front due to gradients created by the shock's electron thermal conduction precursor. Both effects appear to be potentially observable in experiments at laser facilities. We re-examine published studies of magnetogenesis in galaxy cluster formation, and conclude that the simulations in question had inadequate resolution to reliably estimate the field generation rate. Corrected estimates suggest primordial field values in the range $B\sim 10^{-22}$G --- $10^{-19}$G by $z=3$.

astro-ph.GA↗