arXiv subjects

Peter K. G. Williams

Publications and source records attributed to Peter K. G. Williams.

At least 37 records · Page 2Linked to original sources

arXiv2023

Characterization Of Inpaint Residuals In Interferometric Measurements of the Epoch Of Reionization

Radio Frequency Interference (RFI) is one of the systematic challenges preventing 21cm interferometric instruments from detecting the Epoch of Reionization. To mitigate the effects of RFI on data analysis pipelines, numerous inpaint techniques have been developed to restore RFI corrupted data. We examine the qualitative and quantitative errors introduced into the visibilities and power spectrum due to inpainting. We perform our analysis on simulated data as well as real data from the Hydrogen Epoch of Reionization Array (HERA) Phase 1 upper limits. We also introduce a convolutional neural network that capable of inpainting RFI corrupted data in interferometric instruments. We train our network on simulated data and show that our network is capable at inpainting real data without requiring to be retrained. We find that techniques that incorporate high wavenumbers in delay space in their modeling are best suited for inpainting over narrowband RFI. We also show that with our fiducial parameters Discrete Prolate Spheroidal Sequences (DPSS) and CLEAN provide the best performance for intermittent ``narrowband'' RFI while Gaussian Progress Regression (GPR) and Least Squares Spectral Analysis (LSSA) provide the best performance for larger RFI gaps. However we caution that these qualitative conclusions are sensitive to the chosen hyperparameters of each inpainting technique. We find these results to be consistent in both simulated and real visibilities. We show that all inpainting techniques reliably reproduce foreground dominated modes in the power spectrum. Since the inpainting techniques should not be capable of reproducing noise realizations, we find that the largest errors occur in the noise dominated delay modes. We show that in the future, as the noise level of the data comes down, CLEAN and DPSS are most capable of reproducing the fine frequency structure in the visibilities of HERA data.

astro-ph.IM↗

arXiv2023

Search for the Epoch of Reionisation with HERA: Upper Limits on the Closure Phase Delay Power Spectrum

Radio interferometers aiming to measure the power spectrum of the redshifted 21 cm line during the Epoch of Reionisation (EoR) need to achieve an unprecedented dynamic range to separate the weak signal from overwhelming foreground emissions. Calibration inaccuracies can compromise the sensitivity of these measurements to the effect that a detection of the EoR is precluded. An alternative to standard analysis techniques makes use of the closure phase, which allows one to bypass antenna-based direction-independent calibration. Similarly to standard approaches, we use a delay spectrum technique to search for the EoR signal. Using 94 nights of data observed with Phase I of the Hydrogen Epoch of Reionization Array (HERA), we place approximate constraints on the 21 cm power spectrum at $z=7.7$. We find at 95% confidence that the 21 cm EoR brightness temperature is $\le$(372)$^2$ "pseudo" mK$^2$ at 1.14 "pseudo" $h$ Mpc$^{-1}$, where the "pseudo" emphasises that these limits are to be interpreted as approximations to the actual distance scales and brightness temperatures. Using a fiducial EoR model, we demonstrate the feasibility of detecting the EoR with the full array. Compared to standard methods, the closure phase processing is relatively simple, thereby providing an important independent check on results derived using visibility intensities, or related.

astro-ph.CO↗

arXiv2023

Improved Constraints on the 21 cm EoR Power Spectrum and the X-Ray Heating of the IGM with HERA Phase I Observations

We report the most sensitive upper limits to date on the 21 cm epoch of reionization power spectrum using 94 nights of observing with Phase I of the Hydrogen Epoch of Reionization Array (HERA). Using similar analysis techniques as in previously reported limits (HERA Collaboration 2022a), we find at 95% confidence that $Δ^2(k = 0.34$ $h$ Mpc$^{-1}$) $\leq 457$ mK$^2$ at $z = 7.9$ and that $Δ^2 (k = 0.36$ $h$ Mpc$^{-1}) \leq 3,496$ mK$^2$ at $z = 10.4$, an improvement by a factor of 2.1 and 2.6 respectively. These limits are mostly consistent with thermal noise over a wide range of $k$ after our data quality cuts, despite performing a relatively conservative analysis designed to minimize signal loss. Our results are validated with both statistical tests on the data and end-to-end pipeline simulations. We also report updated constraints on the astrophysics of reionization and the cosmic dawn. Using multiple independent modeling and inference techniques previously employed by HERA Collaboration (2022b), we find that the intergalactic medium must have been heated above the adiabatic cooling limit at least as early as $z = 10.4$, ruling out a broad set of so-called "cold reionization" scenarios. If this heating is due to high-mass X-ray binaries during the cosmic dawn, as is generally believed, our result's 99% credible interval excludes the local relationship between soft X-ray luminosity and star formation and thus requires heating driven by evolved low-metallicity stars.

astro-ph.CO↗

arXiv2023

Impact of instrument and data characteristics in the interferometric reconstruction of the 21 cm power spectrum

Combining the visibilities measured by an interferometer to form a cosmological power spectrum is a complicated process. In a delay-based analysis, the mapping between instrumental and cosmological space is not a one-to-one relation. Instead, neighbouring modes contribute to the power measured at one point, with their respective contributions encoded in the window functions. To better understand the power measured by an interferometer, we assess the impact of instrument characteristics and analysis choices on these window functions. Focusing on the Hydrogen Epoch of Reionization Array (HERA) as a case study, we find that long-baseline observations correspond to enhanced low-k tails of the window functions, which facilitate foreground leakage, whilst an informed choice of bandwidth and frequency taper can reduce said tails. With simple test cases and realistic simulations, we show that, apart from tracing mode mixing, the window functions help accurately reconstruct the power spectrum estimator of simulated visibilities. The window functions depend strongly on the beam chromaticity, and less on its spatial structure - a Gaussian approximation, ignoring side lobes, is sufficient. Finally, we investigate the potential of asymmetric window functions, down-weighting the contribution of low-k power to avoid foreground leakage. The window functions presented here correspond to the latest HERA upper limits for the full Phase I data. They allow an accurate reconstruction of the power spectrum measured by the instrument and will be used in future analyses to confront theoretical models and data directly in cylindrical space.

astro-ph.CO↗

arXiv2022

A Novel JupyterLab User Experience for Interactive Data Visualization

In the Jupyter ecosystem, data visualization is usually done with "widgets" created as notebook cell outputs. While this mechanism works well in some circumstances, it is not well-suited to presenting interfaces that are long-lived, interactive, and visually rich. Unlike the traditional Jupyter notebook system, the newer JupyterLab application provides a sophisticated extension infrastructure that raises new design possibilities. Here we present a novel user experience (UX) for interactive data visualization in JupyterLab that is based on an "app" that runs alongside the user's notebooks, rather than widgets that are bound inside them. We have implemented this UX for the AAS WorldWide Telescope (WWT) visualization tool. JupyterLab's messaging APIs allow the app to smoothly exchange data with multiple computational kernels, allowing users to accomplish tasks that are not possible using the widget framework. A new Jupyter server extension allows the frontend to request data from kernels asynchronously over HTTP, enabling interactive exploration of gigapixel-scale imagery in WWT. While we have developed this UX for WWT, the overall design and the server extension are portable to other applications and have the potential to unlock a variety of new user activities that aren't currently possible in "science platform" interfaces.

astro-ph.IM↗

arXiv2022

Direct Optimal Mapping for 21cm Cosmology: A Demonstration with the Hydrogen Epoch of Reionization Array

Motivated by the desire for wide-field images with well-defined statistical properties for 21cm cosmology, we implement an optimal mapping pipeline that computes a maximum likelihood estimator for the sky using the interferometric measurement equation. We demonstrate this direct optimal mapping with data from the Hydrogen Epoch of Reionization (HERA) Phase I observations. After validating the pipeline with simulated data, we develop a maximum likelihood figure-of-merit for comparing four sky models at 166MHz with a bandwidth of 100kHz. The HERA data agree with the GLEAM catalogs to <10%. After subtracting the GLEAM point sources, the HERA data discriminate between the different continuum sky models, providing most support for the model of Byrne et al. 2021. We report the computation cost for mapping the HERA Phase I data and project the computation for the HERA 320-antenna data; both are feasible with a modern server. The algorithm is broadly applicable to other interferometers and is valid for wide-field and non-coplanar arrays.

astro-ph.CO↗

arXiv2022

Figure and Figure Caption Extraction for Mixed Raster and Vector PDFs: Digitization of Astronomical Literature with OCR Features

Scientific articles published prior to the "age of digitization" in the late 1990s contain figures which are "trapped" within their scanned pages. While progress to extract figures and their captions has been made, there is currently no robust method for this process. We present a YOLO-based method for use on scanned pages, post-Optical Character Recognition (OCR), which uses both grayscale and OCR-features. When applied to the astrophysics literature holdings of the Astrophysics Data System (ADS), we find F1 scores of 90.9% (92.2%) for figures (figure captions) with the intersection-over-union (IOU) cut-off of 0.9 which is a significant improvement over other state-of-the-art methods.

astro-ph.IM↗

arXiv2022

The First Short GRB Millimeter Afterglow: The Wide-Angled Jet of the Extremely Energetic SGRB 211106A

We present the discovery of the first millimeter afterglow of a short-duration $γ$-ray burst (SGRB) and the first confirmed afterglow of an SGRB localized by the GUANO system on Swift. Our Atacama Large Millimeter/Sub-millimeter Array (ALMA) detection of SGRB 211106A establishes an origin in a faint host galaxy detected in Hubble Space Telescope (HST) imaging at $0.7\lesssim z\lesssim1.4$. From the lack of a detectable optical afterglow, coupled with the bright millimeter counterpart, we infer a high extinction, $A_{\rm V}\gtrsim2.6$ mag along the line of sight, making this the one of the most highly dust-extincted SGRBs known to date. The millimeter-band light curve captures the passage of the synchrotron peak from the afterglow forward shock and reveals a jet break at $t_{\rm jet}=29.2^{+4.5}_{-4.0}$~days. For a presumed redshift of $z=1$, we infer an opening angle, $θ_{\rm jet}=(15.5\pm1.4)$~degrees, and beaming-corrected kinetic energy of $\log(E_{\rm K}/{\rm erg})=51.8\pm0.3$, making this one of the widest and most energetic SGRB jets known to date. Combining all published millimeter-band upper limits in conjunction with the energetics for a large sample of SGRBs, we find that energetic outflows in high density environments are more likely to have detectable millimeter counterparts. Concerted afterglow searches with ALMA should yield detection fractions of 24-40% on timescales of $\gtrsim2$~days at rates $\approx0.8$-1.6 per year, outpacing the historical discovery rate of SGRB centimeter-band afterglows.

astro-ph.HE↗

arXiv2022

Automated Detection of Antenna Malfunctions in Large-N Interferometers: A Case Study with the Hydrogen Epoch of Reionization Array

We present a framework for identifying and flagging malfunctioning antennas in large radio interferometers. We outline two distinct categories of metrics designed to detect outliers along known failure modes of large arrays: cross-correlation metrics, based on all antenna pairs, and auto-correlation metrics, based solely on individual antennas. We define and motivate the statistical framework for all metrics used, and present tailored visualizations that aid us in clearly identifying new and existing systematics. We implement these techniques using data from 105 antennas in the Hydrogen Epoch of Reionization Array (HERA) as a case study. Finally, we provide a detailed algorithm for implementing these metrics as flagging tools on real data sets.

astro-ph.IM↗

arXiv2021

The Galactic Faraday rotation sky 2020

This work gives an update to existing reconstructions of the Galactic Faraday rotation sky by processing almost all Faraday rotation data sets available at the end of the year 2020. Observations of extra-Galactic sources in recent years have, among other regions, further illuminated the previously under-constrained southern celestial sky, as well as parts of the inner disc of the Milky Way. This has culminated in an all-sky data set of 55,190 data points, which is a significant expansion on the 41,330 used in previous works, hence making an updated separation of the Galactic component a promising venture. The increased source density allows us to present our results in a resolution of about $1.3\cdot 10^{-2}\, \mathrm{deg}^2$ ($46.8\,\mathrm{arcmin}^2$), which is a twofold increase compared to previous works. As for previous Faraday rotation sky reconstructions, this work is based on information field theory, a Bayesian inference scheme for field-like quantities which handles noisy and incomplete data. In contrast to previous reconstructions, we find a significantly thinner and pronounced Galactic disc with small-scale structures exceeding values of several thousand $\mathrm{rad}\,\mathrm{m}^{-2}$. The improvements can mainly be attributed to the new catalog of Faraday data, but are also supported by advances in correlation structure modeling within numerical information field theory. We furthermore give a detailed discussion on statistical properties of the Faraday rotation sky and investigate correlations to other data sets.

astro-ph.GA↗

arXiv2021

A Real Time Processing System for Big Data in Astronomy: Applications to HERA

As current- and next-generation astronomical instruments come online, they will generate an unprecedented deluge of data. Analyzing these data in real time presents unique conceptual and computational challenges, and their long-term storage and archiving is scientifically essential for generating reliable, reproducible results. We present here the real-time processing (RTP) system for the Hydrogen Epoch of Reionization Array (HERA), a radio interferometer endeavoring to provide the first detection of the highly redshifted 21 cm signal from Cosmic Dawn and the Epoch of Reionization by an interferometer. The RTP system consists of analysis routines run on raw data shortly after they are acquired, such as calibration and detection of radio-frequency interference (RFI) events. RTP works closely with the Librarian, the HERA data storage and transfer manager which automatically ingests data and transfers copies to other clusters for post-processing analysis. Both the RTP system and the Librarian are public and open source software, which allows for them to be modified for use in other scientific collaborations. When fully constructed, HERA is projected to generate over 50 terabytes (TB) of data each night, and the RTP system enables the successful scientific analysis of these data.

astro-ph.IM↗

arXiv2021

First Results from HERA Phase I: Upper Limits on the Epoch of Reionization 21 cm Power Spectrum

We report upper-limits on the Epoch of Reionization (EoR) 21 cm power spectrum at redshifts 7.9 and 10.4 with 18 nights of data ($\sim36$ hours of integration) from Phase I of the Hydrogen Epoch of Reionization Array (HERA). The Phase I data show evidence for systematics that can be largely suppressed with systematic models down to a dynamic range of $\sim10^9$ with respect to the peak foreground power. This yields a 95% confidence upper limit on the 21 cm power spectrum of $Δ^2_{21} \le (30.76)^2\ {\rm mK}^2$ at $k=0.192\ h\ {\rm Mpc}^{-1}$ at $z=7.9$, and also $Δ^2_{21} \le (95.74)^2\ {\rm mK}^2$ at $k=0.256\ h\ {\rm Mpc}^{-1}$ at $z=10.4$. At $z=7.9$, these limits are the most sensitive to-date by over an order of magnitude. While we find evidence for residual systematics at low line-of-sight Fourier $k_\parallel$ modes, at high $k_\parallel$ modes we find our data to be largely consistent with thermal noise, an indicator that the system could benefit from deeper integrations. The observed systematics could be due to radio frequency interference, cable sub-reflections, or residual instrumental cross-coupling, and warrant further study. This analysis emphasizes algorithms that have minimal inherent signal loss, although we do perform a careful accounting in a companion paper of the small forms of loss or bias associated with the pipeline. Overall, these results are a promising first step in the development of a tuned, instrument-specific analysis pipeline for HERA, particularly as Phase II construction is completed en route to reaching the full sensitivity of the experiment.

astro-ph.CO↗

arXiv2021

Effects of model incompleteness on the drift-scan calibration of radio telescopes

Precision calibration poses challenges to experiments probing the redshifted 21-cm signal of neutral hydrogen from the Cosmic Dawn and Epoch of Reionization (z~30-6). In both interferometric and global signal experiments, systematic calibration is the leading source of error. Though many aspects of calibration have been studied, the overlap between the two types of instruments has received less attention. We investigate the sky based calibration of total power measurements with a HERA dish and an EDGES style antenna to understand the role of auto-correlations in the calibration of an interferometer and the role of sky in calibrating a total power instrument. Using simulations we study various scenarios such as time variable gain, incomplete sky calibration model, and primary beam model. We find that temporal gain drifts, sky model incompleteness, and beam inaccuracies cause biases in the receiver gain amplitude and the receiver temperature estimates. In some cases, these biases mix spectral structure between beam and sky resulting in spectrally variable gain errors. Applying the calibration method to the HERA and EDGES data, we find good agreement with calibration via the more standard methods. Although instrumental gains are consistent with beam and sky errors similar in scale to those simulated, the receiver temperatures show significant deviations from expected values. While we show that it is possible to partially mitigate biases due to model inaccuracies by incorporating a time-dependent gain model in calibration, the resulting errors on calibration products are larger and more correlated. Completely addressing these biases will require more accurate sky and primary beam models.

astro-ph.CO↗

arXiv2021

A Pilot Radio Search for Magnetic Activity in Directly Imaged Exoplanets

We present the first systematic search for GHz frequency radio emission from directly imaged exoplanets using Very Large Array (VLA) observations of sufficient angular resolution to separate the planets from their host stars. We obtained results for five systems and eight exoplanets located at $\lesssim 50$ pc, through new observations (Ross 458, GU Psc, and 51 Eri) and archival data (GJ 504 and HR 8799). We do not detect radio emission from any of the exoplanets, with $3σ$ luminosity upper limits of $(0.9-23)\times10^{21}$ erg s$^{-1}$. These limits are comparable to the level of radio emission detected in several ultracool dwarfs, including T dwarfs, whose masses are only a factor of two times higher than those of the directly-imaged exoplanets. Despite the lack of detections in this pilot study, we highlight the need for continued GHz frequency radio observations of nearby exoplanets at $μ$Jy-level sensitivity.

astro-ph.EP↗

arXiv2021

Methods of Error Estimation for Delay Power Spectra in $21\,\textrm{cm}$ Cosmology

Precise measurements of the 21 cm power spectrum are crucial for understanding the physical processes of hydrogen reionization. Currently, this probe is being pursued by low-frequency radio interferometer arrays. As these experiments come closer to making a first detection of the signal, error estimation will play an increasingly important role in setting robust measurements. Using the delay power spectrum approach, we have produced a critical examination of different ways that one can estimate error bars on the power spectrum. We do this through a synthesis of analytic work, simulations of toy models, and tests on small amounts of real data. We find that, although computed independently, the different error bar methodologies are in good agreement with each other in the noise-dominated regime of the power spectrum. For our preferred methodology, the predicted probability distribution function is consistent with the empirical noise power distributions from both simulated and real data. This diagnosis is mainly in support of the forthcoming HERA upper limit, and also is expected to be more generally applicable.

astro-ph.IM↗

arXiv2021

Validation of the HERA Phase I Epoch of Reionization 21 cm Power Spectrum Software Pipeline

We describe the validation of the HERA Phase I software pipeline by a series of modular tests, building up to an end-to-end simulation. The philosophy of this approach is to validate the software and algorithms used in the Phase I upper limit analysis on wholly synthetic data satisfying the assumptions of that analysis, not addressing whether the actual data meet these assumptions. We discuss the organization of this validation approach, the specific modular tests performed, and the construction of the end-to-end simulations. We explicitly discuss the limitations in scope of the current simulation effort. With mock visibility data generated from a known analytic power spectrum and a wide range of realistic instrumental effects and foregrounds, we demonstrate that the current pipeline produces power spectrum estimates that are consistent with known analytic inputs to within thermal noise levels (at the 2 sigma level) for k > 0.2 h/Mpc for both bands and fields considered. Our input spectrum is intentionally amplified to enable a strong `detection' at k ~0.2 h/Mpc -- at the level of ~25 sigma -- with foregrounds dominating on larger scales, and thermal noise dominating at smaller scales. Our pipeline is able to detect this amplified input signal after suppressing foregrounds with a dynamic range (foreground to noise ratio) of > 10^7. Our validation test suite uncovered several sources of scale-independent signal loss throughout the pipeline, whose amplitude is well-characterized and accounted for in the final estimates. We conclude with a discussion of the steps required for the next round of data analysis.

astro-ph.IM↗

arXiv2020

Redundant-Baseline Calibration of the Hydrogen Epoch of Reionization Array

In 21 cm cosmology, precision calibration is key to the separation of the neutral hydrogen signal from very bright but spectrally-smooth astrophysical foregrounds. The Hydrogen Epoch of Reionization Array (HERA), an interferometer specialized for 21 cm cosmology and now under construction in South Africa, was designed to be largely calibrated using the self-consistency of repeated measurements of the same interferometric modes. This technique, known as "redundant-baseline calibration" resolves most of the internal degrees of freedom in the calibration problem. It assumes, however, on antenna elements with identical primary beams placed precisely on a redundant grid. In this work, we review the detailed implementation of the algorithms enabling redundant-baseline calibration and report results with HERA data. We quantify the effects of real-world non-redundancy and how they compare to the idealized scenario in which redundant measurements differ only in their noise realizations. Finally, we study how non-redundancy can produce spurious temporal structure in our calibration solutions--both in data and in simulations--and present strategies for mitigating that structure.

astro-ph.IM↗

arXiv2020

Developing a Real Time Processing System for HERA

The Hydrogen Epoch of Reionization Array (HERA) is a radio telescope in the Karoo desert of South Africa endeavoring to observe Cosmic Dawn and the Epoch of Reionization. When fully constructed, it will consist of 350 antennas and generate over 60 terabytes (TB) of data each night. In order to keep pace with the relatively large rate of data, we have developed the real-time processing (RTP) system for HERA. The RTP is responsible for inspecting data for acceptable levels of quality and flagging unusable data, as well as providing an initial calibration solution. The RTP system consists of the pipeline management system, called the hera_opm package, and the data management system, called the librarian. Though the systems have been developed in the context of analyzing HERA data, they feature public and open source software, and have been designed to be adapted and used in other contexts as necessary.

astro-ph.IM↗