SearcharxivSearch

arXiv subjects

Alexander Williamson

Publications and source records attributed to Alexander Williamson.

2 recordsLinked to original sources

Deep Investigation of Neutral Gas Origins (DINGO): Options for robust Deep Spectral Line Imaging in the SKA-Era

The data storage requirements for deep spectral line observations with next-generation radio interferometers like the Australian Square Kilometre Array Pathfinder (ASKAP) and the Square Kilometre Array (SKA) are challenging. The default strategy is to reduce data after each daily observation and stack the resulting images. Although computationally efficient, this approach risks propagating systematic errors (e.g. RFI, continuum and deconvolution residuals) and degrades data quality. Imaging the entire deep dataset jointly, the traditional approach, is prohibitively expensive in storage and compute. We present an alternative \textit{uv}-grid stacking method and compare its outcomes with both the traditional approach, our benchmark, and the default image-stacking method, using 200~h of the Deep Investigation of Neutral Gas Origins (DINGO) pilot and main survey data. Our method pauses the standard imaging pipeline after forming the daily residual visibility grids, which are then stacked and jointly deconvolved to combine many epochs of data. Relative to the traditional method, image-stacking recovers a median of 0.92$_{-0.02}^{+0.05}$ of the reference {\HI} flux across our source sample, and \textit{uv}-grid stacking recovers 0.99$_{-0.04}^{+0.01}$. For the brightest source, both methods show a similar, negligible flux offset of $\sim$3 per~cent from the traditional flux. {\HI} velocity widths ($W_{50}$, $W_{20}$) are recovered to within a few per~cent by both methods, with image-stacking showing somewhat larger deviations and scatter. Image-stacking further introduces non-physical artefacts, such as negative bowls around strong sources, indicating poor deconvolution and loss of physical information. Based on these findings, we intend to apply \textit{uv}-grid stacking to the DINGO survey on ASKAP.

astro-ph.IM

Deep Investigation of Neutral Gas Origins (DINGO): Options for the Processing and Storage of Radio Astronomy Data for robust Deep Spectral Line Imaging in the SKA-Era using uv-Grids

The next generation of radio astronomy telescopes are challenging existing data analysis paradigms, as they have an order of magnitude more antennas and larger bandwidth. Foremost amongst these are deep spectral line surveys, because these have the largest number of epochs and spectral channels per dataset. For example, the Deep Investigation of Neutral Gas Origins (DINGO) project on the Australian Square Kilometre Array Pathfinder (ASKAP) aims to observe over 3,200 hours spread over hundreds of observing sessions, covering two tiles, two footprints and two frequency settings. The two primary problems encountered when processing this data are the need for storage and that processing is primarily I/O limited. To address these issues, we have implemented a deep imaging pipeline based on the storage of an intermediate data product in the software ASKAPsoft, that of the uv-gridded data, and have demonstrated lossy and lossless compression of this data on ASKAP, using MGARD and ADIOS2 libraries. We find data compression ratios from a factor of 7 (lossless) up to 20 (using lossy compression with an absolute error bound of $10^{-4}$), and processing is significantly faster for lossless compression. We discuss the effectiveness of lossy MGARD compression and its adherence to the designated error bounds, the trade-off between these error bounds and the corresponding compression ratios, as well as the potential consequences of these I/O and storage improvements on the science quality of the data products. As lossless compression allows us to achieve the DINGO goals within the storage limitations for the project, this will be the option adopted.

astro-ph.IM