arXiv subjects

Roland Haas

Publications and source records attributed to Roland Haas.

At least 37 records · Page 2Linked to original sources

arXiv2023

HPC-driven computational reproducibility in numerical relativity codes: A use case study with IllinoisGRMHD

Reproducibility of results is a cornerstone of the scientific method. Scientific computing encounters two challenges when aiming for this goal. Firstly, reproducibility should not depend on details of the runtime environment, such as the compiler version or computing environment, so results are verifiable by third-parties. Secondly, different versions of software code executed in the same runtime environment should produce consistent numerical results for physical quantities. In this manuscript, we test the feasibility of reproducing scientific results obtained using the IllinoisGRMHD code that is part of an open-source community software for simulation in relativistic astrophysics, the Einstein Toolkit. We verify that numerical results of simulating a single isolated neutron star with IllinoisGRMHD can be reproduced, and compare them to results reported by the code authors in 2015. We use two different supercomputers: Expanse at SDSC, and Stampede2 at TACC. By compiling the source code archived along with the paper on both Expanse and Stampede2, we find that IllinoisGRMHD reproduces results published in its announcement paper up to errors comparable to round-off level changes in initial data parameters. We also verify that a current version of IlliinoisGRMHD reproduces these results once we account for bug fixes which has occurred since the original publication

gr-qc↗

arXiv2023

Outflows from Short-Lived Neutron-Star Merger Remnants Can Produce a Blue Kilonova

We present a 3D general-relativistic magnetohydrodynamic simulation of a short-lived neutron star remnant formed in the aftermath of a binary neutron star merger. The simulation uses an M1 neutrino transport scheme to track neutrino-matter interactions and is well-suited to studying the resulting nucleosynthesis and kilonova emission. We find that the ejecta in our simulations under-produce $r$-process abundances beyond the second $r$-process peak. For sufficiently long-lived remnants, these outflows \textit{alone} can produce blue kilonovae, including the blue kilonova component observed for AT2017gfo.

astro-ph.HE↗

arXiv2023

A New Moment-Based General-Relativistic Neutrino-Radiation Transport Code: Methods and First Applications to Neutron Star Mergers

We present a new moment-based energy-integrated neutrino transport code for neutron star merger simulations in general relativity. In the merger context, ours is the first code to include Doppler effects at all orders in $\upsilon/c$, retaining all nonlinear neutrino-matter coupling terms. The code is validated with a stringent series of tests. We show that the inclusion of full neutrino-matter coupling terms is necessary to correctly capture the trapping of neutrinos in relativistically moving media, such as in differentially rotating merger remnants. We perform preliminary simulations proving the robustness of the scheme in simulating ab-initio mergers to black hole collapse and long-term neutron star remnants up to ${\sim}70\,$ms. The latter is the longest dynamical spacetime, 3D, general relativistic simulations with full neutrino transport to date. We compare results obtained at different resolutions and using two different closures for the moment scheme. We do not find evidences of significant out-of-thermodynamic equilibrium effects, such as bulk viscosity, on the postmerger dynamics or gravitational wave emission. Neutrino luminosities and average energies are in good agreement with theory expectations and previous simulations by other groups using similar schemes. We compare dynamical and early wind ejecta properties obtained with M1 and with our older neutrino treatment. We find that the M1 results have systematically larger proton fractions. However, the differences in the nucleosynthesis yields are modest. This work sets the basis for future detailed studies spanning a wider set of neutrino reactions, binaries and equations of state.

astro-ph.HE↗

arXiv2023

General relativistic simulations of collapsing binary neutron star mergers with Monte-Carlo neutrino transport

Recent gravitational wave observations of neutron star-neutron star and neutron star-black hole binaries appear to indicate that massive neutron stars may not be too uncommon in merging systems. In this manuscript, we present a first set of evolution of massive neutron star binaries using Monte-Carlo radiation transport for the evolution of neutrinos. We study a range of systems, from nearly symmetric binaries that collapse to a black hole before forming a disk or ejecting material, to more asymmetric binaries in which tidal disruption of the lower mass star leads to the production of more interesting post-merger remnants. For the latter type of systems, we additionally study the impact of viscosity on the properties of the outflows, and compare our results to two recent simulations of identical binaries performed with the WhiskyTHC code. We find agreement on the black hole properties, disk mass, and mass and velocity of the outflows within expected numerical uncertainties, and some minor but noticeable differences in the evolution of the electron fraction when using a subgrid viscosity model, with viscosity playing a more minor role in our simulations. The method used to account for r-process heating in the determination of the outflow properties appears to have a larger impact on our result than those differences between numerical codes. We also use the simulation with the most ejected material to verify that our newly implemented Lagrangian tracers provide a reasonable sampling of the matter outflows as they leave the computational grid. We note that, given the lack of production of hot outflows in these mergers, the main role of neutrinos in these systems is to set the composition of the post-merger remnant. One of the main potential use of our simulations is thus as improved initial conditions for longer evolutions of such remnants.

astro-ph.HE↗

arXiv2023

Numerical relativity higher order gravitational waveforms of eccentric, spinning, non-precessing binary black hole mergers

We use the open source, community-driven, numerical relativity software, the Einstein Toolkit to study the physics of eccentric, spinning, nonprecessing binary black hole mergers with mass-ratios $q=\{2, 4, 6\}$, individual dimensionless spin parameters $χ_{1z}=\pm0.6$, $χ_{2z}=\pm0.3$, that include higher order gravitational wave modes $\ell\leq4$, except for memory modes. Assuming stellar mass binary black hole mergers that may be detectable by the advanced LIGO detectors, we find that including modes up to $\ell=4$ increases the signal-to-noise of compact binaries between $3.5\%$ to $35\%$, compared to signals that only include the $\ell=|m|=2$ mode. We use two waveform models, TEOBResumS and SEOBNRE, which incorporate spin and eccentricity corrections in the waveform dynamics, to quantify the orbital eccentricity of our numerical relativity catalog in a gauge-invariant manner through fitting factor calculations. Our findings indicate that the inclusion of higher order wave modes has a measurable effect in the recovery of moderately and highly eccentric black hole mergers, and thus it is essential to develop waveform models and signal processing tools that accurately describe the physics of these astrophysical sources.

gr-qc↗

arXiv2022

GRaM-X: A new GPU-accelerated dynamical spacetime GRMHD code for Exascale computing with the Einstein Toolkit

We present GRaM-X (General Relativistic accelerated Magnetohydrodynamics on AMReX), a new GPU-accelerated dynamical-spacetime general relativistic magnetohydrodynamics (GRMHD) code which extends the GRMHD capability of Einstein Toolkit to GPU-based exascale systems. GRaM-X supports 3D adaptive mesh refinement (AMR) on GPUs via a new AMR driver for the Einstein Toolkit called CarpetX which in turn leverages AMReX, an AMR library developed for use by the United States DOE's Exascale Computing Project (ECP). We use the Z4c formalism to evolve the equations of GR and the Valencia formulation to evolve the equations of GRMHD. GRaM-X supports both analytic as well as tabulated equations of state. We implement TVD and WENO reconstruction methods as well as the HLLE Riemann solver. We test the accuracy of the code using a range of tests on static spacetime, e.g. 1D MHD shocktubes, the 2D magnetic rotor and a cylindrical explosion, as well as on dynamical spacetimes, i.e. the oscillations of a 3D TOV star. We find excellent agreement with analytic results and results of other codes reported in literature. We also perform scaling tests and find that GRaM-X shows a weak scaling efficiency of $\sim 40-50\%$ on 2304 nodes (13824 NVIDIA V100 GPUs) with respect to single-node performance on OLCF's supercomputer Summit.

astro-ph.IM↗

arXiv2021

Deep Learning with Quantized Neural Networks for Gravitational Wave Forecasting of Eccentric Compact Binary Coalescence

We present the first application of deep learning forecasting for binary neutron stars, neutron star - black hole systems, and binary black hole mergers that span an eccentricity range e <= 0.9. We train neural networks that describe these astrophysical populations, and then test their performance by injecting simulated eccentric signals in advanced LIGO noise available at the \texttt{Gravitational Wave Open Science Center} to: 1) quantify how fast neural networks identify these signals before the binary components merge; 2) quantify how accurately neural networks estimate the time to merger once gravitational waves are identified; and 3) estimate the time-dependent sky localization of these events from early detection to merger. Our findings show that deep learning can identify eccentric signals from a few seconds (for binary black holes) up to tens of seconds (for binary neutron stars) prior to merger. A quantized version of our neural networks achieves 4x reduction in model size, and up to 2.5x inference speed up. These novel algorithms may be used to facilitate time-sensitive multi-messenger astrophysics observations of compact binaries in dense stellar environments.

gr-qc↗

arXiv2021

Source term method for binary neutron stars initial data

The initial condition problem for a binary neutron star system requires a Poisson equation solver for the velocity potential with a Neumann-like boundary condition on the surface of the star. Difficulties that arise in this boundary value problem are: a) the boundary is not known a-priori, but constitutes part of the solution of the problem; b) various terms become singular at the boundary. In this work, we present a new method to solve the fluid Poisson equation for irrotational/spinning binary neutron stars. The advantage of the new method is that it does not require complex fluid surface fitted coordinates and it can be implemented in a Cartesian grid, which is a standard choice in numerical relativity calculations. This is accomplished by employing the source term method proposed by Towers, where the boundary condition is treated as a jump condition and is incorporated as additional source terms in the Poisson equation, which is then solved iteratively. The issue of singular terms caused by vanishing density on the surface is resolved with an additional separation that shifts the computation boundary to the interior of the star. We present two-dimensional tests to show the convergence of the source term method, and we further apply this solver to a realistic three-dimensional binary neutron star problem. By comparing our solution with the one coming from the initial data solver COCAL, we demonstrate agreement to approximately $1\%$. Our method can be used in other problems with non-smooth solutions like in magnetized neutron stars.

gr-qc↗

arXiv2021

Observation of eccentric binary black hole mergers with second and third generation gravitational wave detector networks

[Abridged] We introduce an improved version of the Eccentric, Non-spinning, Inspiral-Gaussian-process Merger Approximant (ENIGMA) waveform model. We find that this ready-to-use model can: (i) produce physically consistent signals when sampling over 1M samples chosen over the $m_{\{1,\,2\}}\in[5M_{\odot},\,50M_{\odot}]$ parameter space, and the entire range of binary inclination angles; (ii) produce waveforms within 0.04 seconds from an initial gravitational wave frequency $f_{\textrm{GW}} =15\,\textrm{Hz}$ and at a sample rate of 8192 Hz; and (iii) reproduce the physics of quasi-circular mergers. We utilize ENIGMA to compute the expected signal-to-noise ratio (SNR) distributions of eccentric binary black hole mergers assuming the existence of second and third generation gravitational wave detector networks that include the twin LIGO detectors, Virgo, KAGRA, LIGO-India, a LIGO-type detector in Australia, Cosmic Explorer, and the Einstein Telescope. In the context of advanced LIGO-type detectors, we find that the SNR of eccentric mergers is always larger than quasi-circular mergers for systems with $e_0\leq0.4$ at $f_{\textrm{GW}} =10\,\textrm{Hz}$, even if the timespan of eccentric signals is just a third of quasi-circular systems with identical total mass and mass-ratio. For Cosmic Explorer-type detector networks, we find that eccentric mergers have similar SNRs than quasi-circular systems for $e_0\leq0.3$ at $f_{\textrm{GW}} =10\,\textrm{Hz}$. Systems with $e_0\sim0.5$ at $f_{\textrm{GW}} =10\,\textrm{Hz}$ have SNRs that range between 50%-90% of the SNR produced by quasi-circular mergers, even if these eccentric signals are just between a third to a tenth the length of quasi-circular systems. For Einstein Telescope-type detectors, we find that eccentric mergers have similar SNRs than quasi-circular systems for $e_0\leq0.4$ at $f_{\textrm{GW}} =5\,\textrm{Hz}$.

gr-qc↗

arXiv2021

DataVault: A Data Storage Infrastructure for the Einstein Toolkit

Data sharing is essential in the numerical simulations research. We introduce a data repository, DataVault, that is designed for data sharing, search and analysis. A comparative study of existing repositories is performed to analyze features that are critical to a data repository. We describe the architecture, workflow, and deployment of DataVault, and provide three use-case scenarios for different communities to facilitate the use and application of DataVault. Potential features are proposed and we outline the future development for these features.

gr-qc↗

arXiv2020

Initial Data and Eccentricity Reduction Toolkit for Binary Black Hole Numerical Relativity Waveforms

The production of numerical relativity waveforms that describe quasicircular binary black hole mergers requires high-quality initial data, and an algorithm to iteratively reduce residual eccentricity. To date, these tools remain closed source, or in commercial software that prevents their use in high performance computing platforms. To address these limitations, and to ensure that the broader numerical relativity community has access to these tools, herein we provide all the required elements to produce high-quality numerical relativity simulations in supercomputer platforms, namely: open source parameter files to numerical simulate spinning black hole binaries with asymmetric mass-ratios; open source $\texttt{Python}$ tools to produce high-quality initial data for numerical relativity simulations of spinning black hole binaries on quasi-circular orbits; open source $\texttt{Python}$ tools for eccentricity reduction, both as stand-alone software and deployed in the $\texttt{Einstein Toolkit}$'s software infrastructure. This open source toolkit fills in a critical void in the literature at a time when numerical relativity has an ever increasing role in the study and interpretation of gravitational wave sources. As part of our community building efforts, and to streamline and accelerate the use of these resources, we provide tutorials that describe, step by step, how to obtain and use these open source numerical relativity tools.

gr-qc↗

arXiv2020

A magnetar engine for short GRBs and kilonovae

We investigate the influence of magnetic fields on the evolution of binary neutron-star (BNS) merger remnants via three-dimensional (3D) dynamical-spacetime general-relativistic (GR) magnetohydrodynamic (MHD) simulations. We evolve a postmerger remnant with an initial poloidal magnetic field, resolve the magnetoturbulence driven by shear flows, and include a microphysical finite-temperature equation of state (EOS). A neutrino leakage scheme that captures the overall energetics and lepton number exchange is also included. We find that turbulence induced by the magnetorotational instability (MRI) in the hypermassive neutron star (HMNS) amplifies magnetic field to beyond magnetar-strength ($10^{15}\, \mathrm{G}$). The ultra-strong toroidal field is able to launch a relativistic jet from the HMNS. We also find a magnetized wind that ejects neutron-rich material with a rate of $\dot{M}_{\mathrm{ej}} \simeq 1 \times10^{-1}\, \mathrm{M_{\odot}\, s^{-1}}$. The total ejecta mass in our simulation is $5\times 10^{-3}\, \mathrm{M_{\odot}}$. This makes the ejecta from the HMNS an important component in BNS mergers and a promising source of $r$-process elements that can power a kilonova. The jet from the HMNS reaches a terminal Lorentz factor of $\sim 5$ in our highest-resolution simulation. The formation of this jet is aided by neutrino-cooling preventing the accretion disk from protruding into the polar region. As neutrino pair-annihilation and radiative processes in the jet (which were not included in the simulations) will boost the Lorentz factor in the jet further, our simulations demonstrate that magnetars formed in BNS mergers are a viable engine for short gamma-ray bursts (sGRBs).

astro-ph.HE↗

arXiv2020

Solving the Bethe-Salpeter equation on massively parallel architectures

The last ten years have witnessed fast spreading of massively parallel computing clusters, from leading supercomputing facilities down to the average university computing center. Many companies in the private sector have undergone a similar evolution. In this scenario, the seamless integration of software and middleware libraries is a key ingredient to ensure portability of scientific codes and guarantees them an extended lifetime. In this work, we describe the integration of the ChASE library, a modern parallel eigensolver, into an existing legacy code for the first-principles computation of optical properties of materials via solution of the Bethe-Salpeter equation for the optical polarization function. Our numerical tests show that, as a result of integrating ChASE and parallelizing the reading routine, the code experiences a remarkable speedup and greatly improved scaling behavior on both multi- and many-core architectures. We demonstrate that such a modernized BSE code will, by fully exploiting parallel computing architectures and file systems, enable domain scientists to accurately study complex material systems that were not accessible before.

cs.CE↗

arXiv2020

Multipolar Effective-One-Body Waveforms for Precessing Binary Black Holes: Construction and Validation

As gravitational-wave detectors become more sensitive, we will access a greater variety of signals emitted by compact binary systems, shedding light on their astrophysical origin and environment. A key physical effect that can distinguish among formation scenarios is the misalignment of the spins with the orbital angular momentum, causing the spins and the binary's orbital plane to precess. To accurately model such systems, it is crucial to include multipoles beyond the dominant quadrupole. Here, we develop the first multipolar precessing waveform model in the effective-one-body (EOB) formalism for the inspiral, merger and ringdown (IMR) of binary black holes: SEOBNRv4PHM. In the nonprecessing limit, the model reduces to SEOBNRv4HM, which was calibrated to numerical-relativity (NR) simulations, and waveforms from perturbation theory. We validate SEOBNRv4PHM by comparing it to the public catalog of 1405 precessing NR waveforms of the Simulating eXtreme Spacetimes (SXS) collaboration, and also to new 118 precessing NR waveforms, which span mass ratios 1-4 and spins up to 0.9. We stress that SEOBNRv4PHM is not calibrated to NR simulations in the precessing sector. We compute the unfaithfulness against the 1523 SXS precessing NR waveforms, and find that, for $94\%$ ($57\%$) of the cases, the maximum value, in the total mass range $20-200 M_\odot$, is below $3\%$ ($1\%$). Those numbers become $83\%$ ($20\%$) when using the IMR, multipolar, precessing phenomenological model IMRPhenomPv3HM. We investigate the impact of such unfaithfulness values with two parameter-estimation studies on synthetic signals. We also compute the unfaithfulness between those waveform models and identify in which part of the parameter space they differ the most. We validate them also against the multipolar, precessing NR surrogate model NRSur7dq4, and find that the SEOBNRv4PHM model outperforms IMRPhenomPv3HM.

gr-qc↗

arXiv2019

Enabling real-time multi-messenger astrophysics discoveries with deep learning

Multi-messenger astrophysics is a fast-growing, interdisciplinary field that combines data, which vary in volume and speed of data processing, from many different instruments that probe the Universe using different cosmic messengers: electromagnetic waves, cosmic rays, gravitational waves and neutrinos. In this Expert Recommendation, we review the key challenges of real-time observations of gravitational wave sources and their electromagnetic and astroparticle counterparts, and make a number of recommendations to maximize their potential for scientific discovery. These recommendations refer to the design of scalable and computationally efficient machine learning algorithms; the cyber-infrastructure to numerically simulate astrophysical sources, and to process and interpret multi-messenger astrophysics data; the management of gravitational wave detections to trigger real-time alerts for electromagnetic and astroparticle follow-ups; a vision to harness future developments of machine learning and cyber-infrastructure resources to cope with the big-data requirements; and the need to build a community of experts to realize the goals of multi-messenger astrophysics.

gr-qc↗

arXiv2019

Physics of eccentric binary black hole mergers: A numerical relativity perspective

Gravitational wave observations of eccentric binary black hole mergers will provide unequivocal evidence for the formation of these systems through dynamical assembly in dense stellar environments. The study of these astrophysically motivated sources is timely in view of electromagnetic observations, consistent with the existence of stellar mass black holes in the globular cluster M22 and in the Galactic center, and the proven detection capabilities of ground-based gravitational wave detectors. In order to get insights into the physics of these objects in the dynamical, strong-field gravity regime, we present a catalog of 89 numerical relativity waveforms that describe binary systems of non-spinning black holes with mass-ratios $1\leq q \leq 10$, and initial eccentricities as high as $e_0=0.18$ fifteen cycles before merger. We use this catalog to quantify the loss of energy and angular momentum through gravitational radiation, and the astrophysical properties of the black hole remnant, including its final mass and spin, and recoil velocity. We discuss the implications of these results for gravitational wave source modeling, and the design of algorithms to search for and identify eccentric binary black hole mergers in realistic detection scenarios.

gr-qc↗

arXiv2019

Fusing numerical relativity and deep learning to detect higher-order multipole waveforms from eccentric binary black hole mergers

We determine the mass-ratio, eccentricity and binary inclination angles that maximize the contribution of the higher-order waveform multipoles $(\ell, \, |m|)= \{(2,\,2),\, (2,\,1),\, (3,\,3),\, (3,\,2), \, (3,\,1),\, (4,\,4),\, (4,\,3),\, (4,\,2),\,(4,\,1)\}$ for the gravitational wave detection of eccentric binary black hole mergers. We carry out this study using numerical relativity waveforms that describe non-spinning black hole binaries with mass-ratios $1\leq q \leq 10$, and orbital eccentricities as high as $e_0=0.18$ fifteen cycles before merger. For stellar-mass, asymmetric mass-ratio, binary black hole mergers, and assuming LIGO's Zero Detuned High Power configuration, we find that in regions of parameter space where black hole mergers modeled with $\ell=|m|=2$ waveforms have vanishing signal-to-noise ratios, the inclusion of $(\ell, \, |m|)$ modes enables the observation of these sources with signal-to-noise ratios that range between 30\% to 45\% the signal-to-noise ratio of optimally oriented binary black hole mergers modeled with $\ell=|m|=2$ numerical relativity waveforms. Having determined the parameter space where $(\ell, \, |m|)$ modes are important for gravitational wave detection, we construct waveform signals that describe these astrophysically motivate scenarios, and demonstrate that these topologically complex signals can be detected and characterized in real LIGO noise with deep learning algorithms.

gr-qc↗

arXiv2019

Unequal Mass Binary Neutron Star Simulations with Neutrino Transport: Ejecta and Neutrino Emission

We present twelve new simulations of unequal mass neutron star mergers. The simulations were preformed with the SpEC code, and utilize nuclear-theory based equations of state and a two-moment gray neutrino transport scheme with an improved energy estimate based on evolving the number density. We model the neutron stars with the SFHo, LS220 and DD2 equations of state (EOS) and we study the neutrino and matter emission of all twelve models to search for robust trends between binary parameters and emission characteristics. We find that the total mass of the dynamical ejecta exceeds $0.01M_\odot$ only for SFHo with weak dependence on the mass-ratio across all models. We find that the ejecta have a broad electron fraction ($Y_e$) distribution ($\approx 0.06-0.48$), with mean $0.2$. $Y_e$ increases with neutrino irradiation over time, but decreases with increasing binary asymmetry. We also find that the models have ejecta with a broad asymptotic velocity distribution ($\approx 0.05-0.7c$). The average velocity lies in the range $0.2c - 0.3c$ and decreases with binary asymmetry. Furthermore, we find that disk mass increases with binary asymmetry and stiffness of the EOS. The $Y_e$ of the disk increases with softness of the EOS. The strongest neutrino emission occurs for the models with soft EOS. For (anti) electron neutrinos we find no significant dependence of the magnitude or angular distribution or neutrino luminosity with mass-ratio. The heavier neutrino species have a luminosity dependence on mass-ratio but an angular distribution which does not change with mass-ratio.

gr-qc↗