arXiv subjects

Roland Haas

Publications and source records attributed to Roland Haas.

At least 55 records · Page 3Linked to original sources

arXiv2019

Supporting High-Performance and High-Throughput Computing for Experimental Science

The advent of experimental science facilities-instruments and observatories, such as the Large Hadron Collider, the Laser Interferometer Gravitational Wave Observatory, and the upcoming Large Synoptic Survey Telescope-has brought about challenging, large-scale computational and data processing requirements. Traditionally, the computing infrastructure to support these facility's requirements were organized into separate infrastructure that supported their high-throughput needs and those that supported their high-performance computing needs. We argue that to enable and accelerate scientific discovery at the scale and sophistication that is now needed, this separation between high-performance computing and high-throughput computing must be bridged and an integrated, unified infrastructure provided. In this paper, we discuss several case studies where such infrastructure has been implemented. These case studies span different science domains, software systems, and application requirements as well as levels of sustainability. A further aim of this paper is to provide a basis to determine the common characteristics and requirements of such infrastructure, as well as to begin a discussion of how best to support the computing requirements of existing and future experimental science facilities.

cs.DC↗

arXiv2019

Deep Learning for Multi-Messenger Astrophysics: A Gateway for Discovery in the Big Data Era

This report provides an overview of recent work that harnesses the Big Data Revolution and Large Scale Computing to address grand computational challenges in Multi-Messenger Astrophysics, with a particular emphasis on real-time discovery campaigns. Acknowledging the transdisciplinary nature of Multi-Messenger Astrophysics, this document has been prepared by members of the physics, astronomy, computer science, data science, software and cyberinfrastructure communities who attended the NSF-, DOE- and NVIDIA-funded "Deep Learning for Multi-Messenger Astrophysics: Real-time Discovery at Scale" workshop, hosted at the National Center for Supercomputing Applications, October 17-19, 2018. Highlights of this report include unanimous agreement that it is critical to accelerate the development and deployment of novel, signal-processing algorithms that use the synergy between artificial intelligence (AI) and high performance computing to maximize the potential for scientific discovery with Multi-Messenger Astrophysics. We discuss key aspects to realize this endeavor, namely (i) the design and exploitation of scalable and computationally efficient AI algorithms for Multi-Messenger Astrophysics; (ii) cyberinfrastructure requirements to numerically simulate astrophysical sources, and to process and interpret Multi-Messenger Astrophysics data; (iii) management of gravitational wave detections and triggers to enable electromagnetic and astro-particle follow-ups; (iv) a vision to harness future developments of machine and deep learning and cyberinfrastructure resources to cope with the scale of discovery in the Big Data Era; (v) and the need to build a community that brings domain experts together with data scientists on equal footing to maximize and accelerate discovery in the nascent field of Multi-Messenger Astrophysics.

astro-ph.IM↗

arXiv2019

Time-symmetry, symplecticity and stability of Euler-Maclaurin and Lanczos-Dyche integration

Numerical evolution of time-dependent differential equations via explicit Runge-Kutta or Taylor methods typically fails to preserve symmetries of a system. It is known that there exists no numerical integration method that in general preserves both the energy and the symplectic structure of a Hamiltonian system. One is thus normally forced to make a choice. Nevertheless, a symmetric integration formula, obtained by Lanczos-Dyche via two-point Taylor expansion (or Hermite interpolation), is shown here to preserve both energy as well as symplectic structure for linear systems. This formula shares similarities with the Euler-Maclaurin formula, but is superconvergent rather than asymptotically convergent. For partial differential equations, the resulting evolution methods are unconditionally stable, i.e, not subject to a Courant-Friedrichs-Lewy limit. Although generally implicit, these methods become explicit for linear systems.

math.NA↗

arXiv2018

Gravitational waveforms from SpEC simulations : neutron star-neutron star and low-mass black hole-neutron star binaries

Gravitational waveforms from numerical simulations are a critical tool to test and analytically calibrate the waveform models used to study the properties of merging compact objects. In this paper, we present a series of high-accuracy waveforms produced with the SpEC code for systems involving at least one neutron star. We provide for the first time waveforms with sub-radian accuracy over more than twenty cycles for low-mass black hole-neutron star binaries, including binaries with non-spinning objects, and binaries with rapidly spinning neutron stars that maximize the impact on the gravitational wave signal of the near-resonant growth of the fundamental excitation mode of the neutron star (f-mode). We also provide for the first time with SpEC a high-accuracy neutron star-neutron star waveform. These waveforms are made publicly available as part of the SxS catalogue. We compare our results to analytical waveform models currently implemented in data analysis pipelines. For most simulations, the models lie outside of the predicted numerical errors in the last few orbits before merger, but do not show systematic deviations from the numerical results: comparing different models appears to provide reasonable estimates of the modeling errors. The sole exception is the equal-mass simulation using a rapidly counter-rotating neutron star to maximize the impact of the excitation of the f-mode, for which all models perform poorly. This is however expected, as even the single model that takes f-mode excitation into account ignores the significant impact of the neutron star spin on the f-mode excitation frequency.

gr-qc↗

arXiv2018

Container solutions for HPC Systems: A Case Study of Using Shifter on Blue Waters

Software container solutions have revolutionized application development approaches by enabling lightweight platform abstractions within the so-called "containers." Several solutions are being actively developed in attempts to bring the benefits of containers to high-performance computing systems with their stringent security demands on the one hand and fundamental resource sharing requirements on the other. In this paper, we discuss the benefits and short-comings of such solutions when deployed on real HPC systems and applied to production scientific applications.We highlight use cases that are either enabled by or significantly benefit from such solutions. We discuss the efforts by HPC system administrators and support staff to support users of these type of workloads on HPC systems not initially designed with these workloads in mind focusing on NCSA's Blue Waters system.

cs.DC↗

arXiv2018

A large scale dynamo and magnetoturbulence in rapidly rotating core-collapse supernovae

Magnetohydrodynamic (MHD) turbulence is of key importance in many high-energy astrophysical systems, where MHD instabilities can amplify local magnetic field over very short time scales. Specifically, the magnetorotational instability (MRI) and dynamo action have been suggested as a mechanism to grow magnetar-strength magnetic field ($\ge10^{15} G$) and magnetorotationally power the explosion of a rotating massive star. Such stars are progenitor candidates for type Ic-bl hypernova explosions and make up all supernovae connected to long gamma-ray bursts (GRBs). The MRI has been studied with local high-resolution shearing box simulations in 3D or with global 2D simulations, but it is an open question whether MRI-driven turbulence can result in the creation of a large-scale ordered and dynamically relevant field. Here we report results from global 3D general-relativistic magnetohydrodynamic (GRMHD) turbulence simulations and show that MRI-driven MHD turbulence in rapidly rotating protoneutron stars produces an inverse cascade of energy. We find a large-scale ordered toroidal field that is consistent with the formation of bipolar magnetorotationally driven outflows. Our results demonstrate that rapidly rotating massive stars are plausible progenitors for both type Ic-bl supernovae and long GRBs, present a viable formation scenario for magnetars, and may account for potentially magnetar-powered superluminous supernovae.

astro-ph.HE↗

arXiv2018

Eccentric, nonspinning, inspiral, Gaussian-process merger approximant for the detection and characterization of eccentric binary black hole mergers

We present $\texttt{ENIGMA}$, a time domain, inspiral-merger-ringdown waveform model that describes non-spinning binary black holes systems that evolve on moderately eccentric orbits. The inspiral evolution is described using a consistent combination of post-Newtonian theory, self-force and black hole perturbation theory. Assuming eccentric binaries that circularize prior to coalescence, we smoothly match the eccentric inspiral with a stand-alone, quasi-circular merger, which is constructed using machine learning algorithms that are trained with quasi-circular numerical relativity waveforms. We show that $\texttt{ENIGMA}$ reproduces with excellent accuracy the dynamics of quasi-circular compact binaries. We validate $\texttt{ENIGMA}$ using a set of $\texttt{Einstein Toolkit}$ eccentric numerical relativity waveforms, which describe eccentric binary black hole mergers with mass-ratios between $1 \leq q \leq 5.5$, and eccentricities $e_0 \lesssim 0.2$ ten orbits before merger. We use this model to explore in detail the physics that can be extracted with moderately eccentric, non-spinning binary black hole mergers. We use $\texttt{ENIGMA}$ to show that GW150914, GW151226, GW170104, GW170814 and GW170608 can be effectively recovered with spinning, quasi-circular templates if the eccentricity of these events at a gravitational wave frequency of 10Hz satisfies $e_0\leq \{0.175,\, 0.125,\,0.175,\,0.175,\, 0.125\}$, respectively. We show that if these systems have eccentricities $e_0\sim 0.1$ at a gravitational wave frequency of 10Hz, they can be misclassified as quasi-circular binaries due to parameter space degeneracies between eccentricity and spin corrections. Using our catalog of eccentric numerical relativity simulations, we discuss the importance of including higher-order waveform multipoles in gravitational wave searches of eccentric binary black hole mergers.

gr-qc↗

arXiv2017

R-process Nucleosynthesis from Three-Dimensional Magnetorotational Core-Collapse Supernovae

We investigate r-process nucleosynthesis in three-dimensional (3D) general-relativistic magnetohydrodynamic simulations of rapidly rotating strongly magnetized core collapse. The simulations include a microphysical finite-temperature equation of state and a leakage scheme that captures the overall energetics and lepton number exchange due to postbounce neutrino emission and absorption. We track the composition of the ejected material using the nuclear reaction network SkyNet. Our results show that the 3D dynamics of magnetorotational core-collapse supernovae (CCSN) are important for their nucleosynthetic signature. We find that production of r-process material beyond the second peak is reduced by a factor of 100 when the magnetorotational jets produced by the rapidly rotating core undergo a kink instability. Our results indicate that 3D magnetorotationally powered CCSNe are a robust r-process source only if they are obtained by the collapse of cores with unrealistically large precollapse magnetic fields of order $10^{13}$G. Additionally, a comparison simulation that we restrict to axisymmetry, results in overly optimistic r-process production for lower magnetic field strengths.

astro-ph.HE↗

arXiv2017

Python Open Source Waveform Extractor (POWER): An open source, Python package to monitor and post-process numerical relativity simulations

Numerical simulations of Einstein's field equations provide unique insights into the physics of compact objects moving at relativistic speeds, and which are driven by strong gravitational interactions. Numerical relativity has played a key role to firmly establish gravitational wave astrophysics as a new field of research, and it is now paving the way to establish whether gravitational wave radiation emitted from compact binary mergers is accompanied by electromagnetic and astro-particle counterparts. As numerical relativity continues to blend in with routine gravitational wave data analyses to validate the discovery of gravitational wave events, it is essential to develop open source tools to streamline these studies. Motivated by our own experience as users and developers of the open source, community software, the Einstein Toolkit, we present an open source, Python package that is ideally suited to monitor and post-process the data products of numerical relativity simulations, and compute the gravitational wave strain at future null infinity in high performance environments. We showcase the application of this new package to post-process a large numerical relativity catalog and extract higher-order waveform modes from numerical relativity simulations of eccentric binary black hole mergers and neutron star mergers. This new software fills a critical void in the arsenal of tools provided by the Einstein Toolkit Consortium to the numerical relativity community.

gr-qc↗

arXiv2017

Evolution of the Magnetized, Neutrino-Cooled Accretion Disk in the Aftermath of a Black Hole Neutron Star Binary Merger

Black hole-torus systems from compact binary mergers are possible engines for gamma-ray bursts (GRBs). During the early evolution of the post-merger remnant, the state of the torus is determined by a combination of neutrino cooling and magnetically-driven heating processes, so realistic models must include both effects. In this paper, we study the post-merger evolution of a magnetized black hole-neutron star binary system using the Spectral Einstein Code (SpEC) from an initial post-merger state provided by previous numerical relativity simulations. We use a finite-temperature nuclear equation of state and incorporate neutrino effects in a leakage approximation. To achieve the needed accuracy, we introduce improvements to SpEC's implementation of general-relativistic magnetohydrodynamics (MHD), including the use of cubed-sphere multipatch grids and an improved method for dealing with supersonic accretion flows where primitive variable recovery is difficult. We find that a seed magnetic field triggers a sustained source of heating, but its thermal effects are largely cancelled by the accretion and spreading of the torus from MHD-related angular momentum transport. The neutrino luminosity peaks at the start of the simulation, and then drops significantly over the first 20\,ms but in roughly the same way for magnetized and nonmagnetized disks. The heating rate and disk's luminosity decrease much more slowly thereafter. These features of the evolution are insensitive to grid structure and resolution, formulation of the MHD equations, and seed field strength, although turbulent effects are not fully converged

astro-ph.HE↗

arXiv2017

BOSS-LDG: A Novel Computational Framework that Brings Together Blue Waters, Open Science Grid, Shifter and the LIGO Data Grid to Accelerate Gravitational Wave Discovery

We present a novel computational framework that connects Blue Waters, the NSF-supported, leadership-class supercomputer operated by NCSA, to the Laser Interferometer Gravitational-Wave Observatory (LIGO) Data Grid via Open Science Grid technology. To enable this computational infrastructure, we configured, for the first time, a LIGO Data Grid Tier-1 Center that can submit heterogeneous LIGO workflows using Open Science Grid facilities. In order to enable a seamless connection between the LIGO Data Grid and Blue Waters via Open Science Grid, we utilize Shifter to containerize LIGO's workflow software. This work represents the first time Open Science Grid, Shifter, and Blue Waters are unified to tackle a scientific problem and, in particular, it is the first time a framework of this nature is used in the context of large scale gravitational wave data analysis. This new framework has been used in the last several weeks of LIGO's second discovery campaign to run the most computationally demanding gravitational wave search workflows on Blue Waters, and accelerate discovery in the emergent field of gravitational wave astrophysics. We discuss the implications of this novel framework for a wider ecosystem of Higher Performance Computing users.

cs.DC↗

arXiv2017

General Relativistic Three-Dimensional Multi-Group Neutrino Radiation-Hydrodynamics Simulations of Core-Collapse Supernovae

We report on a set of long-term general-relativistic three-dimensional (3D) multi-group (energy-dependent) neutrino-radiation hydrodynamics simulations of core-collapse supernovae. We employ a full 3D two-moment scheme with the local M1 closure, three neutrino species, and 12 energy groups per species. With this, we follow the post-core-bounce evolution of the core of a nonrotating $27$-$M_\odot$ progenitor in full unconstrained 3D and in octant symmetry for $\gtrsim$$ 380\,\mathrm{ms}$. We find the development of an asymmetric runaway explosion in our unconstrained simulation. We test the resolution dependence of our results and, in agreement with previous work, find that low resolution artificially aids explosion and leads to an earlier runaway expansion of the shock. At low resolution, the octant and full 3D dynamics are qualitatively very similar, but at high resolution, only the full 3D simulation exhibits the onset of explosion.

astro-ph.HE↗

arXiv2017

Complete waveform model for compact binaries on eccentric orbits

We present a time domain waveform model that describes the inspiral-merger-ringdown (IMR) of compact binary systems whose components are non-spinning, and which evolve on orbits with low to moderate eccentricity. The inspiral evolution is described using third order post-Newtonian equations both for the equations of motion of the binary, and its far-zone radiation field. This latter component also includes instantaneous, tails and tails-of-tails contributions, and a contribution due to non-linear memory. This framework reduces to the post-Newtonian approximant TaylorT4 at third post-Newtonian order in the zero eccentricity limit. To improve phase accuracy, we incorporate higher-order post-Newtonian corrections for the energy flux of quasi-circular binaries and gravitational self-force corrections to the binding energy of compact binaries. This enhanced inspiral evolution prescription is combined with an analytical prescription for the merger-ringdown evolution using a catalog of numerical relativity simulations. This IMR waveform model reproduces effective-one-body waveforms for systems with mass-ratios between 1 to 15 in the zero eccentricity limit. Using a set of eccentric numerical relativity simulations, not used during calibration, we show that our eccentric model accurately reproduces the features of eccentric compact binary coalescence throughout the merger. Using this model we show that the gravitational wave transients GW150914 and GW151226 can be effectively recovered with template banks of quasi-circular, spin-aligned waveforms if the eccentricity $e_0$ of these systems when they enter the aLIGO band at a gravitational wave frequency of 14 Hz satisfies $e_0^{\rm GW150914}\leq0.15$ and $e_0^{\rm GW151226}\leq0.1$.

gr-qc↗

arXiv2016

Simulations of inspiraling and merging double neutron stars using the Spectral Einstein Code

We present results on the inspiral, merger, and post-merger evolution of a neutron star - neutron star (NSNS) system. Our results are obtained using the hybrid pseudospectral-finite volume Spectral Einstein Code (SpEC). To test our numerical methods, we evolve an equal-mass system for $\approx 22$ orbits before merger. This waveform is the longest waveform obtained from fully general-relativistic simulations for NSNSs to date. Such long (and accurate) numerical waveforms are required to further improve semi-analytical models used in gravitational wave data analysis, for example the effective one body models. We discuss in detail the improvements to SpEC's ability to simulate NSNS mergers, in particular mesh refined grids to better resolve the merger and post-merger phases. We provide a set of consistency checks and compare our results to NSNS merger simulations with the independent BAM code. We find agreement between them, which increases confidence in results obtained with either code. This work paves the way for future studies using long waveforms and more complex microphysical descriptions of neutron star matter in SpEC.

gr-qc↗

arXiv2016

Neutrino-Driven Convection in Core-Collapse Supernovae: High-Resolution Simulations

We present results from high-resolution semi-global simulations of neutrino-driven convection in core-collapse supernovae. We employ an idealized setup with parametrized neutrino heating/cooling and nuclear dissociation at the shock front. We study the internal dynamics of neutrino-driven convection and its role in re-distributing energy and momentum through the gain region. We find that even if buoyant plumes are able to locally transfer heat up to the shock, convection is not able to create a net positive energy flux and overcome the downwards transport of energy from the accretion flow. Turbulent convection does, however, provide a significant effective pressure support to the accretion flow as it favors the accumulation of energy, mass and momentum in the gain region. We derive an approximate equation that is able to explain and predict the shock evolution in terms of integrals of quantities such as the turbulent pressure in the gain region or the effects of non-radial motion of the fluid. We use this relation as a way to quantify the role of turbulence in the dynamics of the accretion shock. Finally, we investigate the effects of grid resolution, which we change by a factor 20 between the lowest and highest resolution. Our results show that the shallow slopes of the turbulent kinetic energy spectra reported in previous studies are a numerical artefact. Kolmogorov scaling is progressively recovered as the resolution is increased.

astro-ph.HE↗

arXiv2016

Gravitational waveforms for neutron star binaries from binary black hole simulations

Gravitational waves from binary neutron star (BNS) and black hole/neutron star (BHNS) inspirals are primary sources for detection by the Advanced Laser Interferometer Gravitational-Wave Observatory. The tidal forces acting on the neutron stars induce changes in the phase evolution of the gravitational waveform, and these changes can be used to constrain the nuclear equation of state. Current methods of generating BNS and BHNS waveforms rely on either computationally challenging full 3D hydrodynamical simulations or approximate analytic solutions. We introduce a new method for computing inspiral waveforms for BNS/BHNS systems by adding the post-Newtonian (PN) tidal effects to full numerical simulations of binary black holes (BBHs), effectively replacing the nontidal terms in the PN expansion with BBH results. Comparing a waveform generated with this method against a full hydrodynamical simulation of a BNS inspiral yields a phase difference of $<1$ radian over $\sim 15$ orbits. The numerical phase accuracy required of BNS simulations to measure the accuracy of the method we present here is estimated as a function of the tidal deformability parameter $λ$.

gr-qc↗

arXiv2016

Low mass binary neutron star mergers : gravitational waves and neutrino emission

Neutron star mergers are among the most promising sources of gravitational waves for advanced ground-based detectors. These mergers are also expected to power bright electromagnetic signals, in the form of short gamma-ray bursts, infrared/optical transients, and radio emission. Simulations of these mergers with fully general relativistic codes are critical to understand the merger and post-merger gravitational wave signals and their neutrinos and electromagnetic counterparts. In this paper, we employ the SpEC code to simulate the merger of low-mass neutron star binaries (two $1.2M_\odot$ neutron stars) for a set of three nuclear-theory based, finite temperature equations of state. We show that the frequency peaks of the post-merger gravitational wave signal are in good agreement with predictions obtained from simulations using a simpler treatment of gravity. We find, however, that only the fundamental mode of the remnant is excited for long periods of time: emission at the secondary peaks is damped on a millisecond timescale in the simulated binaries. For such low-mass systems, the remnant is a massive neutron star which, depending on the equation of state, is either permanently stable or long-lived. We observe strong excitations of l=2, m=2 modes, both in the massive neutron star and in the form of hot, shocked tidal arms in the surrounding accretion torus. We estimate the neutrino emission of the remnant using a neutrino leakage scheme and, in one case, compare these results with a gray two-moment neutrino transport scheme. We confirm the complex geometry of the neutrino emission, also observed in previous simulations with neutrino leakage, and show explicitly the presence of important differences in the neutrino luminosity, disk composition, and outflow properties between the neutrino leakage and transport schemes.

astro-ph.HE↗

arXiv2015

Binary Neutron Stars with Arbitrary Spins in Numerical Relativity

We present a code to construct initial data for binary neutron star systems in which the stars are rotating. Our code, based on a formalism developed by Tichy, allows for arbitrary rotation axes of the neutron stars and is able to achieve rotation rates near rotational breakup. We compute the neutron star angular momentum through quasi-local angular momentum integrals. When constructing irrotational binary neutron stars, we find a very small residual dimensionless spin of $\sim 2\times 10^{-4}$. Evolutions of rotating neutron star binaries show that the magnitude of the stars' angular momentum is conserved, and that the spin- and orbit-precession of the stars is well described by post-Newtonian approximation. We demonstrate that orbital eccentricity of the binary neutron stars can be controlled to $\sim 0.1\%$. The neutron stars show quasi-normal mode oscillations at an amplitude which increases with the rotation rate of the stars.

gr-qc↗