arXiv subjects

François Bonnarel

Publications and source records attributed to François Bonnarel.

17 recordsLinked to original sources

arXiv2022

Annotating TAP responses on-the-fly against an IVOA data model

With the success and widespread of the IVOA Table Access Protocol (1) for discovering and querying tabular data in astronomy, more than one hundred of TAP services exposing altogether 22 thousands of tables are accessible from the IVOA Registries at the time of writing. Currently the TAP protocol presents table data and metadata via a {TAP\_SCHEMA} describing the served tables with their columns and possible joins between them. We explore here how to add an information layer, so that values within table columns can be gathered and used to populate instances of objects defined in a selected IVOA data model like Photometry, Coords, Measure, Transform or the proposed MANGO container model. This information layer is provided through annotation tags which tell how the columns' values can be interpreted as attributes of instances of that model. Then when a TAP query is processed, our server add-on interprets the ADQL query string and produces on-the-fly, when possible, the TAP response as an annotated VOTable document. The FIELD elements in the table response are mapped to corresponding model elements templated for this service. This has been prototyped in Java, using the VOLLT package library and a template annotation document representing elements from the MANGO data model. This has been exercised on examples based on Vizier and Chandra catalogs.

astro-ph.IM↗

arXiv2021

TAP and the Data Models

The purpose of the "TAP and the Data Models" Bird of Feathers session was to discuss the relevance of enabling TAP services to deal with IVOA standardized data models and to refine the functionalities required to implement such a capability.

astro-ph.IM↗

arXiv2021

Supporting FAIR Principles in the Astrophysics Community: the European Experience

FAIR principles have the intent to act as a guideline for those wishing to enhance the reusability of their data holdings and put specific emphasis on enhancing the ability of machines to automatically find and use the data, in addition to supporting its reuse by individuals. Interoperability, one core of these principles, especially when dealing with automated systems' ability to interface with each other, requires open standards to avoid restrictions that negatively impact the user's experience. Open-ness of standards is best supported when the governance itself is open and includes a wide range of community participation. In this contribution we report our experience with the FAIR principles, interoperable systems and open governance in astrophysics. We report on activities that have matured within the ESCAPE project with a focus on interfacing the EOSC architecture and Interoperability Framework.

astro-ph.IM↗

arXiv2021

Towards a Provenance Management System for Astronomical Observatories

We present here a provenance management system adapted to astronomical projects needs. We collected use cases from various astronomy projects and defined a data model in the ecosystem developed by the IVOA (International Virtual Observatory Alliance). From those use cases, we observed that some projects already have data collections generated and archived, from which the provenance has to be extracted (provenance "on top"), and some projects are building complex pipelines that automatically capture provenance information during the data processing (capture "inside"). Different tools and prototypes have been developed and tested to capture, store, access and visualize the provenance information, which participate to the shaping of a full provenance management system able to handle detailed provenance information.

cs.IT↗

arXiv2021

The Virtual Observatory Ecosystem Facing the European Open Science Cloud

The International Virtual Observatory Alliance (IVOA) has developed and built, in the last two decades, an ecosystem of distributed resources, interoperable and based upon open shared technological standards. In doing so the IVOA has anticipated, putting into practice for the astrophysical domain, the ideas of FAIR-ness of data and service resources and the Open-ness of sharing scientific results, leveraging on the underlying open standards required to fill the above. In Europe, efforts in supporting and developing the ecosystem proposed by the IVOA specifications has been provided by a continuous set of EU funded projects up to current H2020 ESCAPE ESFRI cluster. In the meantime, in the last years, Europe has realised the importance of promoting the Open Science approach for the research communities and started the European Open Science Cloud (EOSC) project to create a distributed environment for research data, services and communities. In this framework the European VO community, had to face the move from the interoperability scenario in the astrophysics domain into a larger audience perspective that includes a cross-domain FAIR approach. Within the ESCAPE project the CEVO Work Package (Connecting ESFRI to EOSC through the VO) has one task to deal with this integration challenge: a challenge where an existing, mature, distributed e-infrastructure has to be matched to a forming, more general architecture. CEVO started its works in the first months of 2019 and has already worked on the integration of the VO Registry into the EOSC e-infrastructure. This contribution reports on the first year and a half of integration activities, that involve applications, services and resources being aware of the VO scenario and compatible with the EOSC architecture.

astro-ph.IM↗

arXiv2021

Practical Provenance in Astronomy

Recently the International Virtual Observatory Alliance (IVOA) released a standard to structure provenance metadata, and several implementations are in development in order to capture, store, access and visualize the provenance of astronomy data products. This BoF will be focused on practical needs for provenance in astronomy. A growing number of projects express the requirement to propose FAIR data (Findable, Accessible, Interoperable and Reusable) and thus manage provenance information to ensure the quality, reliability and trustworthiness of this data. The concepts are in place, but now, applied specifications and practical tools are needed to answer concrete use cases. During this session we discussed which strategies are considered by projects (observatories or data providers) to capture provenance in their context and how a end-user might query the provenance information to enhance her/his data selection and retrieval. The objective was to identify the development of tools and formats now needed to make provenance more practical needed to increase provenance take-up in the astronomical domain.

astro-ph.IM↗

arXiv2020

Radio Astronomy visibility data discovery and access using IVOA standards

Enhancing interoperable data access to radio data has become a science priority within the International Virtual Observatory Alliance (IVOA). This lead to the foundation of the IVOA Radio astronomy Interest Group. Several radio astronomers and project scientists enrolled in various projects (NRAO, ASKAP, LOFAR, JIVE, ALMA, SKA, INAF, NenuFAR, etc.) have joined. Together they are paving the way to a better integration of their services in the virtual observatory (VO) infrastructure and propose extension of IVOA standards to help achieving this goal. Calibrated radio datasets such as cubes, images, spectra and time series can already be searched and retrieved using the ObsCore/ObsTAP specification defined in the IVOA, or by data product-specific services like SIAv2, SODA, SSA and ConeSearch. However, properties of radio visibility data are not fully implemented in the VO landscape yet. We need specific features to refine data discovery and selection that are adapted to radio astronomers' need. In this context the VO team at the Centre de Données astronomiques de Strasbourg (CDS) proposes to consider the ObsCore/ObsTAP specification and to establish cross-walks between the ObsCore and the existing Measurement Set (MS) metadata profile for data discovery of radio visibility data (VD). In order to account for the difference in granularity between radio VD datasets and science-ready datasets of the VO, the approach splits a MS data file into a list of datasets served by an ObsTAP service, thus enabling coarse grain discovery in the multi-wavelength context. Radio specific metadata such as number of antennae, frequency ranges, $uv$ plane coverage plots, frequency-phase and frequency-amplitude plots, primary and synthesized beams are also provided either by adding column metadata or by using the DataLink technique.

astro-ph.IM↗

arXiv2020

Implementation feedback of the IVOA Provenance data model

The IVOA Provenance Data model defines entities, agents and activities as container classes to describe the provenance of datasets, with the executed tasks and responsibilities attached to agents. It also provides a set of classes to describe the activities type and their configuration template, as well as the configuration applied effectively during the execution of a task. Here we highlight lessons learned in the implementation of the CDS ProvHiPS service distributing provenance metadata for the HST HiPS data collections, and for the HST archive original images used to produce the HiPS tiles. ProvHiPS is based on the ProvTAP protocol, the emerging TAP standard for distributing provenance metadata. ProvTAP queries may rapidly become very complex. Various graph representation strategies, including ad hoc solutions, triplestore and SQL CTE have been considered and are discussed shortly.

astro-ph.IM↗

arXiv2019

Integrating the VO Framework in the EOSC

The European Open Science Cloud (EOSC) is in its early stages, but already some aspects of the EOSC vision are starting to become reality, for example the EOSC portal and the development of metadata catalogues. In the astrophysical domain already exists an open approach to science data: the Virtual Observatory view put in place by the International Virtual Observatory Alliance (IVOA) architecture of standards. The ESCAPE (European Science Cluster of Astronomy & Particle physics ESFRI research infrastructures) project has, among its tasks, to demonstrate that the VO architecture can be integrated within the EOSC building one and to provide guidelines to ESFRI partners (European Strategy Forum on Research Infrastructures) in doing this. This contribution reports on the progress of this integration after the first months of work inside ESCAPE.

astro-ph.IM↗

arXiv2018

Provenance tools for Astronomy

In the context of astronomy projects, scientists have been confronted with the problem of describing in a standardized way how their data have been produced. As presented in a talk at last year's ADASS, the International Virtual Observatory Alliance (IVOA) is working on the definition of a Provenance Data Model, compatible with the W3C PROV model, which shall describe how provenance metadata can be modeled, stored and exchanged in astronomy. In this poster, we present the current status of our developments of libraries and tools, mainly open source, which implement the IVOA Provenance Data Model in order to produce, serve, load and visualize provenance information. These implementations are also needed to validate and adjust the data model and the standard definitions for accessing provenance. The provenance tools developed and created for the W3C framework are reused and extended when possible to tackle the domain of astronomical data.

astro-ph.IM↗

arXiv2018

Provenance as a requirement for large-scale complex astronomical instruments

We developed several pieces of software to enable the tracking of provenance information for the large-scale complex astronomical observatory CTA, the Cherenkov Telescope Array. Such major facilities produce data that will be publicly released to a large community of scientists. There are thus strong requirements to ensure data quality, reliability and trustworthiness. Among those requirements, traceability and reproducibility of the data products have to be included in the development of large projects. Those requirements can be answered by structuring and storing the provenance information for each data product. We followed the Provenance data model, currently discussed at the IVOA, and implemented solutions to collect provenance information during the CTA data processing and the execution of jobs on a work cluster.

astro-ph.IM↗

arXiv2017

IVOA Recommendation: Server-side Operations for Data Access

This document describes the Server-side Operations for Data Access (SODA) web service capability. SODA is a low-level data access capability or server side data processing that can act upon the data files, performing various kinds of operations: filtering/subsection, transformations, pixel operations, and applying functions to the data.

astro-ph.IM↗

arXiv2016

IVOA Data Access Layer: Goals, Achievements and Current Trends

The IVOA Data Access Layer (DAL) working group was created in 2002 to define protocols to homogenize data discovery, data description, data retrieval, and data access processes. We describe its history and status today, and look at current trends for future development of the DAL protocols.

astro-ph.IM↗

arXiv2016

IVOA Provenance data model: hints from the CTA Provenance prototype

We present the last developments on the IVOA Provenance data model, mainly based on the W3C PROV concept. In the context of the Cherenkov astronomy, the data processing stages imply both assumptions and comparison to dedicated simulations. As a consequence, Provenance information is crucial to the end user in order to interpret the high level data products. The Cherenkov Telescope Array (CTA), currently in preparation, is thus a perfect test case for the development of an IVOA standard on Provenance information. We describe general use-cases for the computational Provenance in the CTA production pipeline and explore the proposed W3C notations like PROV-N formats, as well as Provenance access solutions.

astro-ph.IM↗

arXiv2016

IVOA Simple Image Access

The Simple Image Access protocol (SIA) provides capabilities for the discovery, description, access, and retrieval of multi-dimensional image datasets, including 2-D images as well as datacubes of three or more dimensions. SIA data discovery is based on the ObsCore Data Model (ObsCoreDM), which primarily describes data products by the physical axes (spatial, spectral, time, and polarization). Image datasets with dimension greater than 2 are often referred to as datacubes, cube or image cube datasets and may be considered examples of hypercube or n-cube data. In this document the term "image" refers to general multi-dimensional datasets and is synonymous with these other terms unless the image dimensionality is otherwise specified. SIA provides capabilities for image discovery and access. Data discovery and metadata access (using ObsCoreDM) are defined here. The capabilities for drilling down to data files (and related resources) and services for remote access are defined elsewhere, but SIA also allows for direct access to retrieval.

astro-ph.IM↗

arXiv2015

IVOA recommendation: IVOA DataLink

This document describes the linking of data discovery metadata to access to the data itself, further detailed metadata, related resources, and to services that perform operations on the data. The web service capability supports a drill-down into the details of a specific dataset and provides a set of links to the dataset file(s) and related resources. This specification also includes a VOTable-specific method of providing descriptions of one or more services and their input(s), usually using parameter values from elsewhere in the VOTable document. Providers are able to describe services that are relevant to the records (usually datasets with identifiers) by including service descriptors in a result document.

astro-ph.IM↗

arXiv2014

Using the Virtual Observatory: multi-instrument, multi-wavelength study of high-energy sources

This paper presents a tutorial explaining the use of Virtual Observatory tools in high energy astrophysics. Most of the tools used in this paper were developed at the Strasbourg astronomical Data Center and we show how they can be applied to conduct a multi-instrument, multi-wavelength analysis of sources detected by the High Energy Stereoscopic System and the Fermi Large Area Telescope. The analysis involves queries of different data catalogs, selection and cross-correlation techniques on multi-waveband images, and the construction of high energy color-color plots and multi-wavelength spectra. The tutorial is publicly available on the website of the European Virtual Observatory project.

astro-ph.IM↗