arXiv subjects

Manuel Delfino

Publications and source records attributed to Manuel Delfino.

4 recordsLinked to original sources

arXiv2020

CosmoHub: Interactive exploration and distribution of astronomical data on Hadoop

We present CosmoHub (https://cosmohub.pic.es), a web application based on Hadoop to perform interactive exploration and distribution of massive cosmological datasets. Recent Cosmology seeks to unveil the nature of both dark matter and dark energy mapping the large-scale structure of the Universe, through the analysis of massive amounts of astronomical data, progressively increasing during the last (and future) decades with the digitization and automation of the experimental techniques. CosmoHub, hosted and developed at the Port d'Informació Científica (PIC), provides support to a worldwide community of scientists, without requiring the end user to know any Structured Query Language (SQL). It is serving data of several large international collaborations such as the Euclid space mission, the Dark Energy Survey (DES), the Physics of the Accelerating Universe Survey (PAUS) and the Marenostrum Institut de Ciències de l'Espai (MICE) numerical simulations. While originally developed as a PostgreSQL relational database web frontend, this work describes the current version of CosmoHub, built on top of Apache Hive, which facilitates scalable reading, writing and managing huge datasets. As CosmoHub's datasets are seldomly modified, Hive it is a better fit. Over 60 TiB of catalogued information and $50 \times 10^9$ astronomical objects can be interactively explored using an integrated visualization tool which includes 1D histogram and 2D heatmap plots. In our current implementation, online exploration of datasets of $10^9$ objects can be done in a timescale of tens of seconds. Users can also download customized subsets of data in standard formats generated in few minutes.

astro-ph.IM↗

arXiv2019

Toward a Public MAGIC Gamma-Ray Telescope Legacy Data Portal

The MAGIC telescopes are one of the three major IACTs (Imaging Atmospheric Cherenkov Telescopes) for observation of gamma rays in the TeV regime currently operative. MAGIC functions since 2003, and has published data from more than 60 sources, mostly blazars. MAGIC already provides astronomical \texttt{.fits} files with basic final scientific products such as spectral energy distributions, light curves and skymaps from published results. In future, the format of the files can be complemented with further relevant information to the community: a) by including the full multi-wavelength dataset enclosed in a publication, b) providing data in alternative easy-to-use formats such as ASCII or ECSV, which are accessible with other commonly used packages such as \texttt{astropy} or \texttt{gammapy}. Finally, besides high level products, activities have started to provide photon event lists and instrument response functions in a format such that scientists within and outside the community are allowed to perform higher level analysis. A second aim is to provide a full legacy of MAGIC data. This contribution will illustrate the achievements and plans of this activity.

astro-ph.IM↗

arXiv2019

The Physics of the Accelerating Universe Camera

The PAU (Physics of the Accelerating Universe) Survey goal is to obtain photometric redshifts (photo-z) and Spectral Energy Distribution (SED) of astronomical objects with a resolution roughly one order of magnitude better than current broad band photometric surveys. To accomplish this, a new large field of view camera (PAUCam) has been designed, built, commissioned and is now operated at the William Herschel Telescope (WHT). With the current WHT Prime Focus corrector, the camera covers ~1-degree diameter Field of View (FoV), of which, only the inner ~40 arcmin diameter are unvignetted. The focal plane consists of a mosaic of 18 2k$x4k Hamamatsu fully depleted CCDs, with high quantum efficiency up to 1 micrometers in wavelength. To maximize the detector coverage within the FoV, filters are placed in front of the CCDs inside the camera cryostat (made out of carbon fiber) using a challenging movable tray system. The camera uses a set of 40 narrow band filters ranging from ~4500 to ~8500 Angstroms complemented with six standard broad-band filters, ugrizY. The PAU Survey aims to cover roughly 100 square degrees over fields with existing deep photometry and galaxy shapes to obtain accurate photometric redshifts for galaxies down to i_AB~22.5, detecting also galaxies down to i_AB~24 with less precision in redshift. With this data set we will be able to measure intrinsic alignments, galaxy clustering and perform galaxy evolution studies in a new range of densities and redshifts. Here, we describe the PAU camera, its first commissioning results and performance.

astro-ph.IM↗

arXiv2018

The PAU Survey: Operation and orchestration of multi-band survey data

The Physics of the Accelerating Universe (PAU) Survey is an international project for the study of cosmological parameters associated with Dark Energy. PAU's 18-CCD camera (PAUCam), installed at the prime focus of the William Herschel Telescope at the Roque de los Muchachos Observatory (La Palma, Canary Islands), scans part of the northern sky, to collect low resolution spectral information of millions of galaxies with its unique set of 40 narrow-band filters in the optical range from 450 nm to 850 nm, and a set of 6 standard broad band filters. The PAU data management (PAUdm) team is in charge of treating the data, including data transfer from the observatory to the PAU Survey data center, hosted at Port d'Informació Científica (PIC). PAUdm is also in charge of the storage, data reduction and, finally, of making the results available to the scientific community. We describe the technical solutions adopted to cover different aspects of the PAU Survey data management, from the computing infrastructure to support the operations, to the software tools and web services for the data process orchestration and exploration. In particular we will focus on the PAU database, developed for the coordination of the different PAUdm tasks, and to preserve and guarantee the consistency of data and metadata.

astro-ph.IM↗