arXiv subjects

Gregory P. Dubois-Felsmann

Publications and source records attributed to Gregory P. Dubois-Felsmann.

11 recordsLinked to original sources

arXiv2026

The SPHEREx Image and Spectrophotometry Processing Pipeline

In this paper, we describe the SPHEREx image and spectrophotometry data processing pipeline, an infrastructure and software system designed to produce calibrated spectral images and photometric measurements for NASA's SPHEREx mission. SPHEREx is carrying out a series of four all-sky spectrophotometric surveys at 6.15 arcsecond resolution in 102 spectral channels spanning 0.75 to 5 microns. The pipeline which will deliver the flux- and wavelength-calibrated data products deriving from these surveys has been developed and is operated by the SPHEREx Science Data Center at Caltech/IPAC in collaboration with the SPHEREx Science Team. Here we describe the framework and modules used in the pipeline, along with the data products, which are available at the NASA/IPAC Infrared Science Archive.

astro-ph.IM↗

arXiv2025

The SPHEREx Satellite Mission

SPHEREx, a NASA explorer satellite launched on 11 March 2025, is carrying out the first all-sky near-infrared spectral survey. The satellite observes in 102 spectral bands from 0.75 to 5.0 um with a resolving power ranging from 35 to 130 in 6.2 arcsecond pixels. The observatory obtains a 5-sigma depth of 19.5 - 19.9 AB mag for 0.75 to 3.8 um and 17.8 - 18.8 AB mag for 3.8 to 5.0 um after mapping the full sky four times over two years. Scientifically, SPHEREx will produce a large galaxy redshift survey over the full sky, intended to constrain the amplitude of inflationary non-Gaussianity. The observations will produce two deep spectral maps near the ecliptic poles that will use intensity mapping to probe the evolution of galaxies over cosmic history. By mapping the depth of infrared absorption features over the Galactic plane, SPHEREx will comprehensively survey the abundance and composition of water and other biogenic ice species in the interstellar medium. The initial data are rapidly released in the form of spectral images to the public. The project will release specialized data products over the life of the mission as the surveys proceed. The science team will also produce specialized spectral catalogs on planet-bearing and low-mass stars, solar system objects, and galaxy clusters 3 years after launch. We describe the design of the instrument and spacecraft, which flow from the core science requirements. Finally, we present an initial evaluation of the in-flight performance and key characteristics.

astro-ph.IM↗

arXiv2024

Implementing SIAv2 Over Rubin Observatory's Data Butler

The IVOA Simple Image Access version 2 protocol defines an easy way to provide community access to a collection of data. At the Vera C. Rubin Observatory we currently enable ObsTAP access to our data holdings via an ObsCore export or view of our Data Butler repositories. This approach comes with some deployment constraints, such as requiring pgsphere and compatibility with our CADC TAP implementation, so recently we decided to see whether we could instead provide an SIAv2 service that talks directly to our Data Butler. Here we describe our motivation, implementation strategies, and current deployment status, as well as discussing some metadata mismatches between the Butler data models and SIAv2.

astro-ph.IM↗

arXiv2024

Using Felis to Represent the Semantics and Metadata of Astronomical Data Catalogs

The Data Management team of the Vera C. Rubin Observatory has developed a data description language and toolset, Felis, for defining the semantics and metadata of its public-facing data catalogs. Felis uses a rich Pydantic data model for describing and validating catalog metadata, expressed as a human-readable and editable YAML format. Felis also provides a Python library and command line interface for working with these data models. The metadata is used to populate the TAP_SCHEMA tables for the IVOA TAP services utilized by the Rubin Science Platform (RSP). Felis's current capabilities will be discussed along with some future plans.

astro-ph.IM↗

arXiv2023

The Future of Astronomical Data Infrastructure: Meeting Report

The astronomical community is grappling with the increasing volume and complexity of data produced by modern telescopes, due to difficulties in reducing, accessing, analyzing, and combining archives of data. To address this challenge, we propose the establishment of a coordinating body, an "entity," with the specific mission of enhancing the interoperability, archiving, distribution, and production of both astronomical data and software. This report is the culmination of a workshop held in February 2023 on the Future of Astronomical Data Infrastructure. Attended by 70 scientists and software professionals from ground-based and space-based missions and archives spanning the entire spectrum of astronomical research, the group deliberated on the prevailing state of software and data infrastructure in astronomy, identified pressing issues, and explored potential solutions. In this report, we describe the ecosystem of astronomical data, its existing flaws, and the many gaps, duplication, inconsistencies, barriers to access, drags on productivity, missed opportunities, and risks to the long-term integrity of essential data sets. We also highlight the successes and failures in a set of deep dives into several different illustrative components of the ecosystem, included as an appendix.

astro-ph.IM↗

arXiv2023

Data management and execution systems for the Rubin Observatory Science Pipelines

We present the Rubin Observatory system for data storage/retrieval and pipelined code execution. The layer for data storage and retrieval is named the Butler. It consists of a relational database, known as the registry, to keep track of metadata and relations, and a system to manage where the data is located, named the datastore. Together these systems create an abstraction layer that science algorithms can be written against. This abstraction layer manages the complexities of the large data volumes expected and allows algorithms to be written independently, yet be tied together automatically into a coherent processing pipeline. This system consists of tools which execute these pipelines by transforming them into execution graphs which contain concrete data stored in the Butler. The pipeline infrastructure is designed to be scalable in nature, allowing execution on environments ranging from a laptop all the way up to multi-facility data centers. This presentation will focus on the data management aspects as well as an overview on the creation of pipelines and the corresponding execution graphs.

astro-ph.IM↗

arXiv2022

Software Architecture and System Design of Rubin Observatory

Starting from a description of the Rubin Observatory Data Management System Architecture, and drawing on our experience with and involvement in a range of other projects including Gaia, SDSS, UKIRT, and JCMT, we derive a series of generic design patterns and lessons learned.

astro-ph.IM↗

arXiv2022

The Vera C. Rubin Observatory Data Butler and Pipeline Execution System

The Rubin Observatory's Data Butler is designed to allow data file location and file formats to be abstracted away from the people writing the science pipeline algorithms. The Butler works in conjunction with the workflow graph builder to allow pipelines to be constructed from the algorithmic tasks. These pipelines can be executed at scale using object stores and multi-node clusters, or on a laptop using a local file system. The Butler and pipeline system are now in daily use during Rubin construction and early operations.

astro-ph.IM↗

arXiv2021

Rubin Science Platform on Google: the story so far

We describe Rubin Observatory's experience with offering a data access facility (and associated services including our Science Platform) deployed on Google Cloud infrastructure as part of our pre-Operations Data Preview program.

astro-ph.IM↗

arXiv2018

LSST: from Science Drivers to Reference Design and Anticipated Data Products

(Abridged) We describe here the most ambitious survey currently planned in the optical, the Large Synoptic Survey Telescope (LSST). A vast array of science will be enabled by a single wide-deep-fast sky survey, and LSST will have unique survey capability in the faint time domain. The LSST design is driven by four main science themes: probing dark energy and dark matter, taking an inventory of the Solar System, exploring the transient optical sky, and mapping the Milky Way. LSST will be a wide-field ground-based system sited at Cerro Pachón in northern Chile. The telescope will have an 8.4 m (6.5 m effective) primary mirror, a 9.6 deg$^2$ field of view, and a 3.2 Gigapixel camera. The standard observing sequence will consist of pairs of 15-second exposures in a given field, with two such visits in each pointing in a given night. With these repeats, the LSST system is capable of imaging about 10,000 square degrees of sky in a single filter in three nights. The typical 5$σ$ point-source depth in a single visit in $r$ will be $\sim 24.5$ (AB). The project is in the construction phase and will begin regular survey operations by 2022. The survey area will be contained within 30,000 deg$^2$ with $δ<+34.5^\circ$, and will be imaged multiple times in six bands, $ugrizy$, covering the wavelength range 320--1050 nm. About 90\% of the observing time will be devoted to a deep-wide-fast survey mode which will uniformly observe a 18,000 deg$^2$ region about 800 times (summed over all six bands) during the anticipated 10 years of operations, and yield a coadded map to $r\sim27.5$. The remaining 10\% of the observing time will be allocated to projects such as a Very Deep and Fast time domain survey. The goal is to make LSST data products, including a relational database of about 32 trillion observations of 40 billion objects, available to the public and scientists around the world.

astro-ph↗

arXiv2003

A Generic Multi-node State Monitoring Subsystem

The BaBar online data acquisition (DAQ) system includes approximately fifty Unix systems that collectively implement the level-three trigger. These systems all run the same code. Each of these systems has its own state, and this state is expected to change in response to changes in the overall DAQ system. A specialized subsystem has been developed to initiate processing on this collection of systems, and to monitor them both for error conditions and to ensure that they all follow the same state trajectory within a specifiable period of time. This subsystem receives start commands from the main DAQ run control system, and reports major coherent state changes, as well as error conditions, back to the run control system. This state monitoring subsystem has the novel feature that it does not know anything about the state machines that it is monitoring, and hence does not introduce any fundamentally new state machine into the overall system. This feature makes it trivially applicable to other multi-node subsystems. Indeed it has already found a second application beyond the level-three trigger, within the BaBar experiment.

cs.DC↗