arXiv subjects

Zachariah B. Etienne

Publications and source records attributed to Zachariah B. Etienne.

At least 19 recordsLinked to original sources

arXiv2026

BHaHAHA: A Fast, Robust Apparent Horizon Finder Library for Numerical Relativity

Apparent horizon (AH) finders are essential for characterizing black holes and excising their interiors in numerical relativity (NR) simulations. However, open-source AH finders to date are tightly coupled to individual NR codes. We introduce BHaHAHA, the BlackHoles@Home Apparent Horizon Algorithm, the first open-source, infrastructure-agnostic library for AH finding in NR. BHaHAHA implements the first-ever hyperbolic flow-based approach, recasting the elliptic partial differential equation for a marginally outer trapped surface as a damped nonlinear wave equation. To enhance performance, BHaHAHA incorporates a multigrid-inspired refinement strategy, an over-relaxation technique, and OpenMP parallelization. When compared to a naïve hyperbolic relaxation implementation, these enhancements result in 64x speedups for difficult common-horizon finds on a single spacetime slice, enabling BHaHAHA to achieve runtimes within 10% of the widely used (single-core) AHFinderDirect and outperform it on multiple cores. For dynamic horizon tracking with typical core counts on a high-performance-computing cluster, BHaHAHA is approximately 2.1 times faster than AHFinderDirect at accuracies limited by interpolation of metric data from the host NR code. Implemented and tested in both the Einstein Toolkit and BlackHoles@Home, BHaHAHA demonstrates that hyperbolic relaxation can be a robust, versatile, and performant approach for AH finding.

gr-qc↗

arXiv2026

Ultra-low damping of the translational motion of a composite graphite rod in a magneto-gravitational trap

We demonstrate an ultra-low dissipation, one-dimensional mechanical oscillator formed by levitating a millimeter-scale composite graphite rod in a room-temperature magneto-gravitational trap. The trap's magnetic field geometry, based on a linear quadrupole, eliminates first-order field gradients in the axial direction, yielding a low oscillation frequency with ultra-low eddy-current losses. Direct ring-down measurements under vacuum compare the damping of the vertical and axial motion; while the vertical motion damps in seconds, the axial motion damps with a time constant of over 5 days. Analysis reveals that this dramatic difference in damping is a result of the symmetry of the magnetic field and the anisotropy of the trap strength. The results are remarkably robust, demonstrating a potential platform for inertial and gravitational sensing.

physics.app-ph↗

arXiv2026

GRHayL: a modern, infrastructure-agnostic, extensible library for GRMHD simulations

Interpreting multimessenger signals from neutron stars and black holes requires reliable general relativistic magnetohydrodynamics (GRMHD) simulations across rapidly evolving high-performance computing platforms, yet key algorithms are routinely rewritten within infrastructure-specific numerical relativity codes, hindering verification and reuse. We present the General Relativistic Hydrodynamics Library (GRHayL), a modular, infrastructure-agnostic GR(M)HD library providing conservative-to-primitive recovery, reconstruction, flux/source and induction operators, equations of state, and neutrino leakage through an intuitive interface. GRHayL refactors and extends the mature IllinoisGRMHD code into reusable point- and stencilwise kernels, enabling rapid development and cross-code validation in diverse frameworks, while easing adoption of new microphysics and future accelerators. We implement the same kernels in the Einstein Toolkit (Carpet and Carpetx) and BlackHoles@Home, demonstrating portability with minimal duplication. Validation combines continuous-integration unit tests with cross-infrastructure comparisons of analytic GRMHD Riemann problems, dynamical Tolman-Oppenheimer-Volkoff evolutions, and binary neutron star mergers, showing comparable or improved behavior over legacy IllinoisGRMHD and established Einstein Toolkit codes.

gr-qc↗

arXiv2025

Gravitational-Wave Signatures of Massive Black Hole Formation

Direct-collapse black holes (DCBHs) are an important component of the massive black hole population of the early universe, and their formation and early mergers will be prominent in the data stream of the Laser Interferometer Space Antenna (LISA). However, the population and binary properties of these early black holes are poorly understood, with masses, mass ratios, spins, and orbital eccentricities strongly dependent on the details of their formation, and the properties of the remaining exterior material (baryonic and non-baryonic), which may be substantial to the point of merger. We report on initial work to simulate the formation, collapse, and/or merger of such DCBH regions in order to extract the resulting gravitational-wave signals.

gr-qc↗

arXiv2025

General relativistic hydrodynamics code for dynamical spacetimes with curvilinear coordinates, tabulated equations of state, and neutrino physics

Many astrophysical systems of interest to numerical relativity-such as rapidly rotating stars, black hole accretion disks, and core-collapse supernovae-exhibit near-symmetries. These systems generally consist of a strongly gravitating central object surrounded by an accretion disk, debris, and ejecta. Simulations can efficiently exploit the near-axisymmetry of these systems by reducing the number of points in the angular direction around the near-symmetry axis, enabling efficient simulations over seconds-long timescales with minimal computational expense. In this paper, we introduce GRoovy, a novel code capable of modeling astrophysical systems containing compact objects by solving the equations of general relativistic hydrodynamics (GRHD) in full general relativity using singular curvilinear (spherical-like and cylindrical-like) and Cartesian coordinates. We demonstrate the code's robustness through a battery of challenging GRHD tests, ranging from flat, static spacetimes to curved, dynamical spacetimes. These tests further showcase the code's capabilities in modeling systems with realistic, finite-temperature equations of state and neutrino cooling via a leakage scheme. GRoovy extensively leverages GRHayL, an open-source, modular, and infrastructure-agnostic general relativistic magnetohydrodynamics library built from the highly robust algorithms of IllinoisGRMHD. Long-term simulations of binary neutron star and black hole-neutron star post-merger remnants will benefit greatly from using a future Charm++-parallelized version of GRoovy to study phenomena such as remnant stability, gamma-ray bursts, and nucleosynthesis.

gr-qc↗

arXiv2025

superB/NRPy: Scalable, Task-Based Numerical Relativity for 3G Gravitational Wave Science

Modern gravitational-wave science demands increasingly accurate and computationally intensive numerical relativity (NR) simulations. The Python-based, open-source NRPy framework generates optimized C/C++ code for NR, including the complete NR code BlackHoles@Home (BH@H), which leverages curvilinear coordinates well-suited to many astrophysical scenarios. Historically, BH@H was limited to single-node OpenMP CPU parallelism. To address this, we introduce superB, an open-source extension to NRPy that enables automatic generation of scalable, task-based, distributed-memory Charm++ code from existing BH@H modules. The generated code partitions the structured grids used by NRPy/BH@H, managing communication between them. Its correctness is validated through bit-identical results with the standard OpenMP version on a single node and via a head-on binary black hole simulation in cylindrical-like coordinates, accurately reproducing quasi-normal modes (up to $\ell=8$). The superB/NRPy-generated code demonstrates excellent strong scaling, achieving an $\approx 45$x speedup on 64 nodes (7168 cores) compared to the original single-node OpenMP code for a large 3D vacuum test. This scalable infrastructure benefits demanding simulations and lays the groundwork for future multi-patch grid support, targeting long inspirals, extreme parameter studies, and rapid follow-ups. This infrastructure readily integrates with other NRPy/BH@H-based projects, enabling performant scaling for the general relativistic hydrodynamics code GRoovy, and facilitating future coupling with GPU acceleration via the NRPy-CUDA project.

gr-qc↗

arXiv2025

Accelerating Numerical Relativity with Code Generation: CUDA-enabled Hyperbolic Relaxation

Next-generation gravitational wave detectors such as Cosmic Explorer, the Einstein Telescope, and LISA, demand highly accurate and extensive gravitational wave (GW) catalogs to faithfully extract physical parameters from observed signals. However, numerical relativity (NR) faces significant challenges in generating these catalogs at the required scale and accuracy on modern computers, as NR codes do not fully exploit modern GPU capabilities. In response, we extend NRPy, a Python-based NR code-generation framework, to develop NRPyEllipticGPU -- a CUDA-optimized elliptic solver tailored for the binary black hole (BBH) initial data problem. NRPyEllipticGPU is the first GPU-enabled elliptic solver in the NR community, supporting a variety of coordinate systems and demonstrating substantial performance improvements on both consumer-grade and HPC-grade GPUs. We show that, when compared to a high-end CPU, NRPyEllipticGPU achieves on a high-end GPU up to a sixteenfold speedup in single precision while increasing double-precision performance by a factor of 2--4. This performance boost leverages the GPU's superior parallelism and memory bandwidth to achieve a compute-bound application and enhancing the overall simulation efficiency. As NRPyEllipticGPU shares the core infrastructure common to NR codes, this work serves as a practical guide for developing full, CUDA-optimized NR codes.

gr-qc↗

arXiv2024

Improved Moving-Puncture Techniques for Compact Binary Simulations

To fully unlock the scientific potential of upcoming gravitational wave (GW) interferometers, numerical relativity (NR) simulation accuracy will need to be greatly enhanced. We present three infrastructure-agnostic improvements to the moving-puncture approach for binary black hole (BBH) simulations, aimed at greatly reducing constraint violation and improving GW predictions. Although these improvements were developed within the highly efficient NR code BlackHoles@Home, we demonstrate their effectiveness in the widely-adopted Einstein Toolkit/Carpet AMR framework. Our improvements include a modified Kreiss-Oliger dissipation prescription, a Hamiltonian-constraint-damping adjustment to the BSSN equations, and an extra term to the 1+log lapse evolution equation that slows the development of the sharp lapse feature, which dominates numerical errors in BBH simulations. With minimal increase in computational cost, these improvements greatly reduce GW noise, enabling the extraction of high-order GW modes previously obscured by numerical noise. They also improve convergence properties near and inside the convergent regime, reduce Hamiltonian (momentum) constraint violations in the strong-field region by roughly two (three) orders of magnitude, and in the GW-extraction zone by five (two) orders of magnitude. To promote community adoption, we have open-sourced the improved Einstein Toolkit thorn BaikalVacuum used in this work. Although our focus is on BBH evolutions and the BSSN formulation, these improvements may also benefit compact binary simulations involving matter and other formulations, a focus for future investigations.

gr-qc↗

arXiv2024

Toward 2D Dynamo Models Calibrated by Global 3D Relativistic Accretion Disk Simulations

Two-dimensional models assuming axisymmetry are an economical way to explore the long-term evolution of black hole accretion disks, but they are only realistic if the feedback of the nonaxisymmetric turbulence on the mean momentum and magnetic fields is incorporated. Dynamo terms added to the 2D induction equation should be calibrated to 3D MHD simulations. For generality, the dynamo tensors should be calibrated as functions of local variables rather than explicit functions of spatial coordinates in a particular basis. In this paper, we study the feedback of non-axisymmetric features on the 2D mean fields using a global 3D, relativistic, Cartesian simulation from the IllinoisGRMHD code. We introduce new methods for estimating overall dynamo alpha and turbulent diffusivity effects as well as measures of the dominance of non-axisymmetric components of energies and fluxes within the disk interior. We attempt closure models of the dynamo EMF using least squares fitting, considering both models where coefficient tensors are functions of space and more global, covariant models. None of these models are judged satisfactory, but we are able to draw conclusions on what sorts of generalizations are and are not promising.

astro-ph.HE↗

arXiv2024

Cross-correlation image analysis for real-time particle tracking

Accurately measuring the translations of objects between images is essential in many fields, including biology, medicine, chemistry, and physics. One important application is tracking one or more particles by measuring their apparent displacements in a series of images. Popular methods, such as the center-of-mass, often require idealized scenarios to reach the shot-noise limit of particle tracking and are, therefore, not generally applicable to multiple image types. More general methods, like maximum likelihood estimation, reliably approach the shot-noise limit, but are too computationally intense for use in real-time applications. These limitations are significant, as real-time, shot-noise-limited particle tracking is of paramount importance for feedback control systems. To fill this gap, we introduce a new cross-correlation-based algorithm that approaches shot-noise-limited displacement detection and a GPU-based implementation for real-time image analysis of a single particle.

physics.optics↗

arXiv2023

Addition of tabulated equation of state and neutrino leakage support to IllinoisGRMHD

We have added support for realistic, microphysical, finite-temperature equations of state (EOS) and neutrino physics via a leakage scheme to IllinoisGRMHD, an open-source GRMHD code for dynamical spacetimes in the Einstein Toolkit. These new features are provided by two new, NRPy+-based codes: NRPyEOS, which performs highly efficient EOS table lookups and interpolations, and NRPyLeakage, which implements a new, AMR-capable neutrino leakage scheme in the Einstein Toolkit. We have performed a series of strenuous validation tests that demonstrate the robustness of these new codes, particularly on the Cartesian AMR grids provided by Carpet. Furthermore, we show results from fully dynamical GRMHD simulations of single unmagnetized neutron stars, and magnetized binary neutron star mergers. This new version of IllinoisGRMHD, as well as NRPyEOS and NRPyLeakage, is pedagogically documented in Jupyter notebooks and fully open source. The codes will be proposed for inclusion in an upcoming version of the Einstein Toolkit.

gr-qc↗

arXiv2023

HPC-driven computational reproducibility in numerical relativity codes: A use case study with IllinoisGRMHD

Reproducibility of results is a cornerstone of the scientific method. Scientific computing encounters two challenges when aiming for this goal. Firstly, reproducibility should not depend on details of the runtime environment, such as the compiler version or computing environment, so results are verifiable by third-parties. Secondly, different versions of software code executed in the same runtime environment should produce consistent numerical results for physical quantities. In this manuscript, we test the feasibility of reproducing scientific results obtained using the IllinoisGRMHD code that is part of an open-source community software for simulation in relativistic astrophysics, the Einstein Toolkit. We verify that numerical results of simulating a single isolated neutron star with IllinoisGRMHD can be reproduced, and compare them to results reported by the code authors in 2015. We use two different supercomputers: Expanse at SDSC, and Stampede2 at TACC. By compiling the source code archived along with the paper on both Expanse and Stampede2, we find that IllinoisGRMHD reproduces results published in its announcement paper up to errors comparable to round-off level changes in initial data parameters. We also verify that a current version of IlliinoisGRMHD reproduces these results once we account for bug fixes which has occurred since the original publication

gr-qc↗

arXiv2023

Ameliorating the Courant-Friedrichs-Lewy condition in spherical coordinates: A double FFT filter method for general relativistic MHD in dynamical spacetimes

Numerical simulations of merging compact objects and their remnants form the theoretical foundation for gravitational wave and multi-messenger astronomy. While Cartesian-coordinate-based adaptive mesh refinement is commonly used for simulations, spherical-like coordinates are more suitable for nearly spherical remnants and azimuthal flows due to lower numerical dissipation in the evolution of fluid angular momentum, as well as requiring fewer numbers of computational cells. However, the use of spherical coordinates to numerically solve hyperbolic partial differential equations can result in severe Courant-Friedrichs-Lewy (CFL) stability condition timestep limitations, which can make simulations prohibitively expensive. This paper addresses this issue for the numerical solution of coupled spacetime and general relativistic magnetohydrodynamics evolutions by introducing a double FFT filter and implementing it within the fully MPI-parallelized SphericalNR framework in the Einstein Toolkit. We demonstrate the effectiveness and robustness of the filtering algorithm by applying it to a number of challenging code tests, and show that it passes these tests effectively, demonstrating convergence while also increasing the timestep significantly compared to unfiltered simulations.

gr-qc↗

arXiv2023

Bound Debris Expulsion from Neutron Star Merger Remnants

Many studies have found that neutron star mergers leave a fraction of the stars' mass in bound orbits surrounding the resulting massive neutron star or black hole. This mass is a site of $r-$ process nucleosynthesis and can generate a wind that contributes to a kilonova. However, comparatively little is known about the dynamics determining its mass or initial structure. Here we begin to investigate these questions, starting with the origin of the disk mass. Using tracer particle as well as discretized fluid data from numerical simulations, we identify where in the neutron stars the debris came from, the paths it takes in order to escape from the neutron stars' interiors, and the times and locations at which its orbital properties diverge from those of neighboring fluid elements that end up remaining in the merged neutron star.

astro-ph.HE↗

arXiv2022

Handing off the outcome of binary neutron star mergers for accurate and long-term post-merger simulations

We perform binary neutron star (BNS) merger simulations in full dynamical general relativity with IllinoisGRMHD, on a Cartesian grid with adaptive-mesh refinement. After the remnant black hole has become nearly stationary, the evolution of the surrounding accretion disk on Cartesian grids over long timescales (1s) is suboptimal, as Cartesian coordinates over-resolve the angular coordinates at large distances, and the accreting plasma flows obliquely across coordinate lines dissipating angular momentum artificially from the disk. To address this, we present the Handoff, a set of computational tools that enables the transfer of general relativistic magnetohydrodynamic (GRMHD) and spacetime data from IllinoisGRMHD to HARM3D, a GRMHD code that specializes in modeling black hole accretion disks in static spacetimes over long timescales, making use of general coordinate systems with spherical topology. We demonstrate that the Handoff allows for a smooth and reliable transition of GRMHD fields and spacetime data, enabling us to efficiently and reliably evolve BNS dynamics well beyond merger. We also discuss future plans, which involve incorporating advanced equations of state and neutrino physics into BNS simulations using the \handoff approach.

astro-ph.HE↗

arXiv2022

NRPyLaTeX: A LaTeX interface to computer algebra systems for general relativity

While each computer algebra system (CAS) contains its own unique syntax for inputting mathematical expressions, LaTeX is perhaps the most widespread language for typesetting mathematics. NRPyLaTeX (NL) enables direct LaTeX input of complex tensorial expressions (written in Einstein notation) relevant to general relativity and differential geometry into the SymPy CAS. As SymPy also supports output compatible with the Mathematica and Maple CASs, NL lowers the learning curve for inputting and manipulating tensorial expressions in three widely used CASs. LaTeX however is a typesetting language, and as such is not designed to resolve ambiguities in mathematical expressions. To address this, NL implements a convenient configuration interface that, e.g., defines variables with certain attributes. Configuration commands appear as LaTeX comments, so that entire NL workflows can fit seamlessly into the LaTeX source code of scientific papers without interfering with the rendered mathematical expressions. Further, NL adopts NRPy+'s rigid syntax for indexed symbols (e.g., tensors), which enables NL output to be directly converted into highly optimized C/C++-code kernels using NRPy+. Finally NL has robust and user-friendly error-handling, which catches common tensor indexing errors and reports unresolved ambiguities, further expediting the input and validation of LaTeX expressions into a CAS.

gr-qc↗

arXiv2022

NRPyElliptic: A Fast Hyperbolic Relaxation Elliptic Solver for Numerical Relativity, I: Conformally Flat, Binary Puncture Initial Data

We introduce NRPyElliptic, an elliptic solver for numerical relativity (NR) built within the NRPy+ framework. As its first application, NRPyElliptic sets up conformally flat, binary black hole (BBH) puncture initial data (ID) on a single numerical domain, similar to the widely used TwoPunctures code. Unlike TwoPunctures, NRPyElliptic employs a hyperbolic relaxation scheme, whereby arbitrary elliptic PDEs are trivially transformed into a hyperbolic system of PDEs. As consumers of NR ID generally already possess expertise in solving hyperbolic PDEs, they will generally find NRPyElliptic easier to tweak and extend than other NR elliptic solvers. When evolved forward in (pseudo)time, the hyperbolic system exponentially reaches a steady state that solves the elliptic PDEs. Notably NRPyElliptic accelerates the relaxation waves, which makes it many orders of magnitude faster than the usual constant-wavespeed approach. While it is still ${\sim}12$x slower than TwoPunctures at setting up full-3D BBH ID, NRPyElliptic requires only ${\approx}0.3\%$ of the runtime for a full BBH simulation in the Einstein Toolkit. Future work will focus on improving performance and generating other types of ID, such as binary neutron star.

gr-qc↗

arXiv2022

HARM3D+NUC: A new method for simulating the post-merger phase of binary neutron star mergers with GRMHD, tabulated EOS and neutrino leakage

The first binary neutron star merger has already been detected in gravitational waves. The signal was accompanied by an electromagnetic counterpart including a kilonova component powered by the decay of radioactive nuclei, as well as a short $γ$-ray burst. In order to understand the radioactively-powered signal, it is necessary to simulate the outflows and their nucleosynthesis from the post-merger disk. Simulating the disk and predicting the composition of the outflows requires general relativistic magnetohydrodynamical (GRMHD) simulations that include a realistic, finite-temperature equation of state (EOS) and self-consistently calculating the impact of neutrinos. In this work, we detail the implementation of a finite-temperature EOS and the treatment of neutrinos in the GRMHD code HARM3D+NUC, based on HARM3D. We include formal tests of both the finite-temperature EOS and the neutrino leakage scheme. We further test the code by showing that, given conditions similar to those of published remnant disks following neutron star mergers, it reproduces both recombination of free nucleons to a neutron-rich composition and excitation of a thermal wind.

astro-ph.HE↗