Searcharxiv⌕ Search

arXiv subjects

Frossie Economou

Publications and source records attributed to Frossie Economou.

At least 19 recordsLinked to original sources
arXiv2026

The Vera C. Rubin Observatory Data Preview 1

We present Rubin Data Preview 1 DP1, the first data from the NSF DOE Vera C Rubin Observatory, comprising raw and calibrated single epoch images, coadds, difference images, detection catalogs, and ancillary data products. DP1 is based on 1792 optical near infrared exposures acquired over 48 distinct nights by the Rubin Commissioning Camera LSSTComCam on the Simonyi Survey Telescope at the Summit Facility on Cerro Pach\'on Chile in late 2024. DP1 covers $\sim$15 deg$^2$ distributed across seven roughly equal-sized non-contiguous fields, each independently observed in six broad photometric bands $ugrizy$. The median FWHM of the point spread function across all bands is approximately 1.14 arcseconds, with the sharpest images reaching about 0.58 arcseconds. The 5$\sigma$ point source depths for coadded images in the deepest field the Extended Chandra Deep Field South are $u$ = 24.55, $g$ = 26.18, $r$ = 25.96, $i$ = 25.71, $z$ = 25.07, $y$ = 23.1. Other fields are no more than 2.2 magnitudes shallower in any band where they have nonzero coverage. DP1 contains approximately 2.3 million distinct astrophysical objects, of which 1.6 million are extended in at least one band in coadds and 431 solar system objects of which 93 are new discoveries. DP1 is approximately 3.5 TB in size and is available to Rubin data rights holders via the Rubin Science Platform a cloud based environment for the analysis of petascale astronomical data. While small compared to future LSST releases its high quality and diversity of data support a broad range of early science investigations ahead of full operations in 2026.

astro-ph.IM↗
arXiv2023

The Future of Astronomical Data Infrastructure: Meeting Report

The astronomical community is grappling with the increasing volume and complexity of data produced by modern telescopes, due to difficulties in reducing, accessing, analyzing, and combining archives of data. To address this challenge, we propose the establishment of a coordinating body, an "entity," with the specific mission of enhancing the interoperability, archiving, distribution, and production of both astronomical data and software. This report is the culmination of a workshop held in February 2023 on the Future of Astronomical Data Infrastructure. Attended by 70 scientists and software professionals from ground-based and space-based missions and archives spanning the entire spectrum of astronomical research, the group deliberated on the prevailing state of software and data infrastructure in astronomy, identified pressing issues, and explored potential solutions. In this report, we describe the ecosystem of astronomical data, its existing flaws, and the many gaps, duplication, inconsistencies, barriers to access, drags on productivity, missed opportunities, and risks to the long-term integrity of essential data sets. We also highlight the successes and failures in a set of deep dives into several different illustrative components of the ecosystem, included as an appendix.

astro-ph.IM↗
arXiv2022

Software Architecture and System Design of Rubin Observatory

Starting from a description of the Rubin Observatory Data Management System Architecture, and drawing on our experience with and involvement in a range of other projects including Gaia, SDSS, UKIRT, and JCMT, we derive a series of generic design patterns and lessons learned.

astro-ph.IM↗
arXiv2021

Rubin Science Platform on Google: the story so far

We describe Rubin Observatory's experience with offering a data access facility (and associated services including our Science Platform) deployed on Google Cloud infrastructure as part of our pre-Operations Data Preview program.

astro-ph.IM↗
arXiv2019

Petabytes to Science

A Kavli foundation sponsored workshop on the theme \emph{Petabytes to Science} was held 12$^{th}$ to 14$^{th}$ of February 2019 in Las Vegas. The aim of the this workshop was to discuss important trends and technologies which may support astronomy. We also tackled how to better shape the workforce for the new trends and how we should approach education and public outreach. This document was coauthored during the workshop and edited in the weeks after. It comprises the discussions and highlights many recommendations which came out of the workshop. We shall distill parts of this document and formulate potential white papers for the decadal survey.

astro-ph.IM↗
arXiv2019

Why is the LSST Science Platform built on Kubernetes?

LSST has chosen Kubernetes as the platform for deploying and operating the LSST Science Platform. We first present the background reasoning behind this decision, including both instrument-agnostic as well as LSST-specific requirements. We then discuss the basic principles of Kubernetes and Helm, and how they are used as the deployment base for the LSST Science Platform. Furthermore, we provide an example of how an external group may use these publicly available software resources to deploy their own instance of the LSST Science Platform, and customize it to their needs. Finally, we discuss how more astronomy software can follow these patterns to gain similar benefits.

astro-ph.IM↗
arXiv2019

The Growing Importance of a Tech Savvy Astronomy and Astrophysics Workforce

Fundamental coding and software development skills are increasingly necessary for success in nearly every aspect of astronomical and astrophysical research as large surveys and high resolution simulations become the norm. However, professional training in these skills is inaccessible or impractical for many members of our community. Students and professionals alike have been expected to acquire these skills on their own, apart from formal classroom curriculum or on-the-job training. Despite the recognized importance of these skills, there is little opportunity to develop them - even for interested researchers. To ensure a workforce capable of taking advantage of the computational resources and the large volumes of data coming in the next decade, we must identify and support ways to make software development training widely accessible to community members, regardless of affiliation or career level. To develop and sustain a technology capable astronomical and astrophysical workforce, we recommend that agencies make funding and other resources available in order to encourage, support and, in some cases, require progress on necessary training, infrastructure and policies. In this white paper, we focus on recommendations for how funding agencies can lead in the promotion of activities to support the astronomy and astrophysical workforce in the 2020s.

astro-ph.IM↗
arXiv2019

The demise of the filesystem and multi level service architecture

Many astronomy data centres still work on filesystems. Industry has moved on; current practice in computing infrastructure is to achieve Big Data scalability using object stores rather than POSIX file systems. This presents us with opportunities for portability and reuse of software underlying processing and archive systems but it also causes problems for legacy implementations in current data centers.

astro-ph.IM↗
arXiv2019

Astro2020 APC White Paper: Astronomy should be in the clouds

Commodity cloud computing, as provided by commercial vendors such as Amazon, Google, and Microsoft, has revolutionized computing in many sectors. With the advent of a new class of big data, public access astronomical facility such as LSST, DKIST, and WFIRST, there exists a real opportunity to combine these missions with cloud computing platforms and fundamentally change the way astronomical data is collected, processed, archived, and curated. Making these changes in a cross-mission, coordinated way can provide unprecedented economies of scale in personnel, data collection and management, archiving, algorithm and software development and, most importantly, science.

astro-ph.IM↗
arXiv2019

Astro2020 APC White Paper: Elevating the Role of Software as a Product of the Research Enterprise

Software is a critical part of modern research, and yet there are insufficient mechanisms in the scholarly ecosystem to acknowledge, cite, and measure the impact of research software. The majority of academic fields rely on a one-dimensional credit model whereby academic articles (and their associated citations) are the dominant factor in the success of a researcher's career. In the petabyte era of astronomical science, citing software and measuring its impact enables academia to retain and reward researchers that make significant software contributions. These highly skilled researchers must be retained to maximize the scientific return from petabyte-scale datasets. Evolving beyond the one-dimensional credit model requires overcoming several key challenges, including the current scholarly ecosystem and scientific culture issues. This white paper will present these challenges and suggest practical solutions for elevating the role of software as a product of the research enterprise.

astro-ph.IM↗
arXiv2018

LSST: from Science Drivers to Reference Design and Anticipated Data Products

(Abridged) We describe here the most ambitious survey currently planned in the optical, the Large Synoptic Survey Telescope (LSST). A vast array of science will be enabled by a single wide-deep-fast sky survey, and LSST will have unique survey capability in the faint time domain. The LSST design is driven by four main science themes: probing dark energy and dark matter, taking an inventory of the Solar System, exploring the transient optical sky, and mapping the Milky Way. LSST will be a wide-field ground-based system sited at Cerro Pachón in northern Chile. The telescope will have an 8.4 m (6.5 m effective) primary mirror, a 9.6 deg$^2$ field of view, and a 3.2 Gigapixel camera. The standard observing sequence will consist of pairs of 15-second exposures in a given field, with two such visits in each pointing in a given night. With these repeats, the LSST system is capable of imaging about 10,000 square degrees of sky in a single filter in three nights. The typical 5$σ$ point-source depth in a single visit in $r$ will be $\sim 24.5$ (AB). The project is in the construction phase and will begin regular survey operations by 2022. The survey area will be contained within 30,000 deg$^2$ with $δ<+34.5^\circ$, and will be imaged multiple times in six bands, $ugrizy$, covering the wavelength range 320--1050 nm. About 90\% of the observing time will be devoted to a deep-wide-fast survey mode which will uniformly observe a 18,000 deg$^2$ region about 800 times (summed over all six bands) during the anticipated 10 years of operations, and yield a coadded map to $r\sim27.5$. The remaining 10\% of the observing time will be allocated to projects such as a Very Deep and Fast time domain survey. The goal is to make LSST data products, including a relational database of about 32 trillion observations of 40 billion objects, available to the public and scientists around the world.

astro-ph↗
arXiv2015

The LSST Data Management System

The Large Synoptic Survey Telescope (LSST) is a large-aperture, wide-field, ground-based survey system that will image the sky in six optical bands from 320 to 1050 nm, uniformly covering approximately $18,000$deg$^2$ of the sky over 800 times. The LSST is currently under construction on Cerro Pachón in Chile, and expected to enter operations in 2022. Once operational, the LSST will explore a wide range of astrophysical questions, from discovering "killer" asteroids to examining the nature of Dark Energy. The LSST will generate on average 15 TB of data per night, and will require a comprehensive Data Management system to reduce the raw data to scientifically useful catalogs and images with minimum human intervention. These reductions will result in a real-time alert stream, and eleven data releases over the 10-year duration of LSST operations. To enable this processing, the LSST project is developing a new, general-purpose, high-performance, scalable, well documented, open source data processing software stack for O/IR surveys. Prototypes of this stack are already capable of processing data from existing cameras (e.g., SDSS, DECam, MegaCam), and form the basis of the Hyper-Suprime Cam (HSC) Survey data reduction pipeline.

astro-ph.IM↗
arXiv2015

Significant problems in FITS limit its use in modern astronomical research

The Flexible Image Transport System (FITS) standard has been a great boon to astronomy, allowing observatories, scientists and the public to exchange astronomical information easily. The FITS standard is, however, showing its age. Developed in the late 1970s the FITS authors made a number of implementation choices for the format that, while common at the time, are now seen to limit its utility with modern data. The authors of the FITS standard could not appreciate the challenges which we would be facing today in astronomical computing. Difficulties we now face include, but are not limited to, having to address the need to handle an expanded range of specialized data product types (data models), being more conducive to the networked exchange and storage of data, handling very large datasets and the need to capture significantly more complex metadata and data relationships. There are members of the community today who find some (or all) of these limitations unworkable, and have decided to move ahead with storing data in other formats. This reaction should be taken as a wakeup call to the FITS community to make changes in the FITS standard, or to see its usage fall. In this paper we detail some selected important problems which exist within the FITS standard today. It is not our intention to prescribe specific remedies to these issues; rather, we hope to call attention of the FITS and greater astronomical computing communities to these issues in the hopes that it will spur action to address them.

astro-ph.IM↗
arXiv2015

Learning from FITS: Limitations in use in modern astronomical research

The Flexible Image Transport System (FITS) standard has been a great boon to astronomy, allowing observatories, scientists and the public to exchange astronomical information easily. The FITS standard, however, is showing its age. Developed in the late 1970s, the FITS authors made a number of implementation choices that, while common at the time, are now seen to limit its utility with modern data. The authors of the FITS standard could not anticipate the challenges which we are facing today in astronomical computing. Difficulties we now face include, but are not limited to, addressing the need to handle an expanded range of specialized data product types (data models), being more conducive to the networked exchange and storage of data, handling very large datasets, and capturing significantly more complex metadata and data relationships. There are members of the community today who find some or all of these limitations unworkable, and have decided to move ahead with storing data in other formats. If this fragmentation continues, we risk abandoning the advantages of broad interoperability, and ready archivability, that the FITS format provides for astronomy. In this paper we detail some selected important problems which exist within the FITS standard today. These problems may provide insight into deeper underlying issues which reside in the format and we provide a discussion of some lessons learned. It is not our intention here to prescribe specific remedies to these issues; rather, it is to call attention of the FITS and greater astronomical computing communities to these problems in the hope that it will spur action to address them.

astro-ph.IM↗
arXiv2014

ORAC-DR: A generic data reduction pipeline infrastructure

ORAC-DR is a general purpose data reduction pipeline system designed to be instrument and observatory agnostic. The pipeline works with instruments as varied as infrared integral field units, imaging arrays and spectrographs, and sub-millimeter heterodyne arrays & continuum cameras. This paper describes the architecture of the pipeline system and the implementation of the core infrastructure. We finish by discussing the lessons learned since the initial deployment of the pipeline system in the late 1990s.

astro-ph.IM↗
arXiv2014

Learning from 25 years of the extensible N-Dimensional Data Format

The extensible N-Dimensional Data Format (NDF) was designed and developed in the late 1980s to provide a data model suitable for use in a variety of astronomy data processing applications supported by the UK Starlink Project. Starlink applications were used extensively, primarily in the UK astronomical community, and form the basis of a number of advanced data reduction pipelines today. This paper provides an overview of the historical drivers for the development of NDF and the lessons learned from using a defined hierarchical data model for many years in data reduction software, data pipelines and in data acquisition systems.

astro-ph.IM↗
arXiv2014

Your data is your dogfood: DevOps in the astronomical observatory

DevOps is the contemporary term for a software development culture that purposefully blurs distinction between software development and IT operations by treating "infrastructure as code." DevOps teams typically implement practices summarised by the colloquial directive to "eat your own dogfood;" meaning that software tools developed by a team should be used internally rather thrown over the fence to operations or users. We present a brief overview of how DevOps techniques bring proven software engineering practices to IT operations. We then discuss the application of these practices to astronomical observatories.

astro-ph.IM↗