arXiv subjects

Andrew J. Connolly

Publications and source records attributed to Andrew J. Connolly.

At least 37 records · Page 2Linked to original sources

arXiv2023

The DECam Ecliptic Exploration Project (DEEP) III: Survey characterization and simulation methods

We present a detailed study of the observational biases of the DECam Ecliptic Exploration Project's (DEEP) B1 data release and survey simulation software that enables direct statistical comparisons between models and our data. We inject a synthetic population of objects into the images, and then subsequently recover them in the same processing as our real detections. This enables us to characterize the survey's completeness as a function of apparent magnitudes and on-sky rates of motion. We study the statistically optimal functional form for the magnitude, and develop a methodology that can estimate the magnitude and rate efficiencies for all survey's pointing groups simultaneously. We have determined that our peak completeness is on average 80\% in each pointing group, and our magnitude drops to $25\%$ of this value at $m_{25} = 26.22$. We describe the freely available survey simulation software and its methodology. We conclude by using it to infer that our effective search area for objects at 40 au is $14.8°^2$, and that our lack of dynamically cold distant objects means that there at most $8\times 10^3$ objects with $60 < a < 80$ au and absolute magnitudes $H \leq 8$.

astro-ph.EP↗

arXiv2023

The DECam Ecliptic Exploration Project (DEEP) VI: first multi-year observations of trans-Neptunian objects

We present the first set of trans-Neptunian objects (TNOs) observed on multiple nights in data taken from the DECam Ecliptic Exploration Project (DEEP). Of these 110 TNOs, 105 do not coincide with previously known TNOs and appear to be new discoveries. Each individual detection for our objects resulted from a digital tracking search at TNO rates of motion, using two to four hour exposure sets, and the detections were subsequently linked across multiple observing seasons. This procedure allows us to find objects with magnitudes $m_{VR} \approx 26$. The object discovery processing also included a comprehensive population of objects injected into the images, with a recovery and linking rate of at least $94\%$. The final orbits were obtained using a specialized orbit fitting procedure that accounts for the positional errors derived from the digital tracking procedure. Our results include robust orbits and magnitudes for classical TNOs with absolute magnitudes $H \sim 10$, as well as a dynamically detached object found at 76 au (semi-major axis $a\approx 77 \, \mathrm{au}$). We find a disagreement between our population of classical TNOs and the CFEPS-L7 three component model for the Kuiper belt.

astro-ph.EP↗

arXiv2023

The DECam Ecliptic Exploration Project (DEEP): V. The Absolute Magnitude Distribution of the Cold Classical Kuiper Belt

The DECam Ecliptic Exploration Project (DEEP) is a deep survey of the trans-Neptunian solar system being carried out on the 4-meter Blanco telescope at Cerro Tololo Inter-American Observatory in Chile using the Dark Energy Camera (DECam). By using a shift-and-stack technique to achieve a mean limiting magnitude of $r \sim 26.2$, DEEP achieves an unprecedented combination of survey area and depth, enabling quantitative leaps forward in our understanding of the Kuiper Belt populations. This work reports results from an analysis of twenty 3 sq.\ deg.\ DECam fields along the invariable plane. We characterize the efficiency and false-positive rates for our moving-object detection pipeline, and use this information to construct a Bayesian signal probability for each detected source. This procedure allows us to treat all of our Kuiper Belt Object (KBO) detections statistically, simultaneously accounting for efficiency and false positives. We detect approximately 2300 candidate sources with KBO-like motion at S/N $>6.5$. We use a subset of these objects to compute the luminosity function of the Kuiper Belt as a whole, as well as the Cold Classical (CC) population. We also investigate the absolute magnitude ($H$) distribution of the CCs, and find consistency with both an exponentially tapered power-law, which is predicted by streaming instability models of planetesimal formation, and a rolling power law. Finally, we provide an updated mass estimate for the Cold Classical Kuiper Belt of $M_{CC}(H_r < 12) = 0.0017^{+0.0010}_{-0.0004} M_{\oplus}$, assuming albedo $p = 0.15$ and density $ρ= 1$ g cm$^{-3}$.

astro-ph.EP↗

arXiv2022

From Data to Software to Science with the Rubin Observatory LSST

The Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) dataset will dramatically alter our understanding of the Universe, from the origins of the Solar System to the nature of dark matter and dark energy. Much of this research will depend on the existence of robust, tested, and scalable algorithms, software, and services. Identifying and developing such tools ahead of time has the potential to significantly accelerate the delivery of early science from LSST. Developing these collaboratively, and making them broadly available, can enable more inclusive and equitable collaboration on LSST science. To facilitate such opportunities, a community workshop entitled "From Data to Software to Science with the Rubin Observatory LSST" was organized by the LSST Interdisciplinary Network for Collaboration and Computing (LINCC) and partners, and held at the Flatiron Institute in New York, March 28-30th 2022. The workshop included over 50 in-person attendees invited from over 300 applications. It identified seven key software areas of need: (i) scalable cross-matching and distributed joining of catalogs, (ii) robust photometric redshift determination, (iii) software for determination of selection functions, (iv) frameworks for scalable time-series analyses, (v) services for image access and reprocessing at scale, (vi) object image access (cutouts) and analysis at scale, and (vii) scalable job execution systems. This white paper summarizes the discussions of this workshop. It considers the motivating science use cases, identified cross-cutting algorithms, software, and services, their high-level technical specifications, and the principles of inclusive collaborations needed to develop them. We provide it as a useful roadmap of needs, as well as to spur action and collaboration between groups and individuals looking to develop reusable software for early LSST science.

astro-ph.IM↗

arXiv2022

MUSSES2020J: The Earliest Discovery of a Fast Blue Ultraluminous Transient at Redshift 1.063

In this Letter, we report the discovery of an ultraluminous fast-evolving transient in rest-frame UV wavelengths, MUSSES2020J, soon after its occurrence by using the Hyper Suprime-Cam (HSC) mounted on the 8.2 m Subaru telescope. The rise time of about 5 days with an extremely high UV peak luminosity shares similarities to a handful of fast blue optical transients whose peak luminosities are comparable with the most luminous supernovae while their timescales are significantly shorter (hereafter "fast blue ultraluminous transient," FBUT). In addition, MUSSES2020J is located near the center of a normal low-mass galaxy at a redshift of 1.063, suggesting a possible connection between the energy source of MUSSES2020J and the central part of the host galaxy. Possible physical mechanisms powering this extreme transient such as a wind-driven tidal disruption event and an interaction between supernova and circumstellar material are qualitatively discussed based on the first multiband early-phase light curve of FBUTs, although whether the scenarios can quantitatively explain the early photometric behavior of MUSSES2020J requires systematical theoretical investigations. Thanks to the ultrahigh luminosity in UV and blue optical wavelengths of these extreme transients, a promising number of FBUTs from the local to the high-z universe can be discovered through deep wide-field optical surveys in the near future.

astro-ph.HE↗

arXiv2022

Machine Learning and Cosmology

Methods based on machine learning have recently made substantial inroads in many corners of cosmology. Through this process, new computational tools, new perspectives on data collection, model development, analysis, and discovery, as well as new communities and educational pathways have emerged. Despite rapid progress, substantial potential at the intersection of cosmology and machine learning remains untapped. In this white paper, we summarize current and ongoing developments relating to the application of machine learning within cosmology and provide a set of recommendations aimed at maximizing the scientific impact of these burgeoning tools over the coming decade through both technical development as well as the fostering of emerging communities.

hep-ph↗

arXiv2021

Sifting Through the Static: Moving Object Detection in Difference Images

Trans-Neptunian Objects (TNOs) provide a window into the history of the Solar System, but they can be challenging to observe due to their distance from the Sun and relatively low brightness. Here we report the detection of 75 moving objects that we could not link to any other known objects, the faintest of which has a VR magnitude of $25.02 \pm 0.93$ using the KBMOD platform. We recover an additional 24 sources with previously-known orbits. We place constraints on the barycentric distance, inclination, and longitude of ascending node of these objects. The unidentified objects have a median barycentric distance of 41.28 au, placing them in the outer Solar System. The observed inclination and magnitude distribution of all detected objects is consistent with previously published KBO distributions. We describe extensions to KBMOD, including a robust percentile-based lightcurve filter, an in-line graphics processing unit (GPU) filter, new coadded stamp generation, and a convolutional neural network (CNN) stamp filter, which allow KBMOD to take advantage of difference images. These enchancements mark a significant improvement in the readiness of KBMOD for deployment on future big data surveys such as LSST.

astro-ph.EP↗

arXiv2021

Optimization of the Observing Cadence for the Rubin Observatory Legacy Survey of Space and Time: a pioneering process of community-focused experimental design

Vera C. Rubin Observatory is a ground-based astronomical facility under construction, a joint project of the National Science Foundation and the U.S. Department of Energy, designed to conduct a multi-purpose 10-year optical survey of the southern hemisphere sky: the Legacy Survey of Space and Time. Significant flexibility in survey strategy remains within the constraints imposed by the core science goals of probing dark energy and dark matter, cataloging the Solar System, exploring the transient optical sky, and mapping the Milky Way. The survey's massive data throughput will be transformational for many other astrophysics domains and Rubin's data access policy sets the stage for a huge potential users' community. To ensure that the survey science potential is maximized while serving as broad a community as possible, Rubin Observatory has involved the scientific community at large in the process of setting and refining the details of the observing strategy. The motivation, history, and decision-making process of this strategy optimization are detailed in this paper, giving context to the science-driven proposals and recommendations for the survey strategy included in this Focus Issue.

astro-ph.IM↗

arXiv2021

THOR: An Algorithm for Cadence-Independent Asteroid Discovery

We present "Tracklet-less Heliocentric Orbit Recovery" (THOR), an algorithm for linking of observations of Solar System objects across multiple epochs that does not require intra-night tracklets or a predefined cadence of observations within a search window. By sparsely covering regions of interest in the phase space with "test orbits", transforming nearby observations over a few nights into the co-rotating frame of the test orbit at each epoch, and then performing a generalized Hough transform on the transformed detections followed by orbit determination (OD) filtering, candidate clusters of observations belonging to the same objects can be recovered at moderate computational cost and little to no constraints on cadence. We validate the effectiveness of this approach by running on simulations as well as on real data from the Zwicky Transient Facility (ZTF). Applied to a short, 2-week, slice of ZTF observations, we demonstrate THOR can recover 97.4% of all previously known and discoverable objects in the targeted ($a > 1.7$ au) population with 5 or more observations and with purity between 97.7% and 100%. This includes 10 likely new discoveries, and a recovery of an $e \sim 1$ comet C/2018 U1 (the comet would have been a ZTF discovery had THOR been running in 2018 when the data were taken). The THOR package and demo Jupyter notebooks are open source and available at https://github.com/moeyensj/thor.

astro-ph.IM↗

arXiv2021

The LSST DESC DC2 Simulated Sky Survey

We describe the simulated sky survey underlying the second data challenge (DC2) carried out in preparation for analysis of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) by the LSST Dark Energy Science Collaboration (LSST DESC). Significant connections across multiple science domains will be a hallmark of LSST; the DC2 program represents a unique modeling effort that stresses this interconnectivity in a way that has not been attempted before. This effort encompasses a full end-to-end approach: starting from a large N-body simulation, through setting up LSST-like observations including realistic cadences, through image simulations, and finally processing with Rubin's LSST Science Pipelines. This last step ensures that we generate data products resembling those to be delivered by the Rubin Observatory as closely as is currently possible. The simulated DC2 sky survey covers six optical bands in a wide-fast-deep (WFD) area of approximately 300 deg^2 as well as a deep drilling field (DDF) of approximately 1 deg^2. We simulate 5 years of the planned 10-year survey. The DC2 sky survey has multiple purposes. First, the LSST DESC working groups can use the dataset to develop a range of DESC analysis pipelines to prepare for the advent of actual data. Second, it serves as a realistic testbed for the image processing software under development for LSST by the Rubin Observatory. In particular, simulated data provide a controlled way to investigate certain image-level systematic effects. Finally, the DC2 sky survey enables the exploration of new scientific ideas in both static and time-domain cosmology.

astro-ph.IM↗

arXiv2020

Learning Spectral Templates for Photometric Redshift Estimation from Broadband Photometry

Estimating redshifts from broadband photometry is often limited by how accurately we can map the colors of galaxies to an underlying spectral template. Current techniques utilize spectrophotometric samples of galaxies or spectra derived from spectral synthesis models. Both of these approaches have their limitations, either the sample sizes are small and often not representative of the diversity of galaxy colors or the model colors can be biased (often as a function of wavelength) which introduces systematics in the derived redshifts. In this paper we learn the underlying spectral energy distributions from an ensemble of $\sim$100K galaxies with measured redshifts and colors. We show that we are able to reconstruct emission and absorption lines at a significantly higher resolution than the broadband filters used to measure the photometry for a sample of 20 spectral templates. We find that our training algorithm reduces the fraction of outliers in the derived photometric redshifts by up to 28%, bias up to 91%, and scatter up to 25%, when compared to estimates using a standard set of spectral templates. We discuss the current limitations of this approach and its applicability for recovering the underlying properties of galaxies. Our derived templates and the code used to produce these results are publicly available in a dedicated Github repository: https://github.com/dirac-institute/photoz_template_learning.

astro-ph.IM↗

arXiv2020

Dimensionality Reduction of SDSS Spectra with Variational Autoencoders

High resolution galaxy spectra contain much information about galactic physics, but the high dimensionality of these spectra makes it difficult to fully utilize the information they contain. We apply variational autoencoders (VAEs), a non-linear dimensionality reduction technique, to a sample of spectra from the Sloan Digital Sky Survey. In contrast to Principal Component Analysis (PCA), a widely used technique, VAEs can capture non-linear relationships between latent parameters and the data. We find that a VAE can reconstruct the SDSS spectra well with only six latent parameters, outperforming PCA with the same number of components. Different galaxy classes are naturally separated in this latent space, without class labels having been given to the VAE. The VAE latent space is interpretable because the VAE can be used to make synthetic spectra at any point in latent space. For example, making synthetic spectra along tracks in latent space yields sequences of realistic spectra that interpolate between two different types of galaxies. Using the latent space to find outliers may yield interesting spectra: in our small sample, we immediately find unusual data artifacts and stars misclassified as galaxies. In this exploratory work, we show that VAEs create compact, interpretable latent spaces that capture non-linear features of the data. While a VAE takes substantial time to train (~1 day for 48000 spectra), once trained, VAEs can enable the fast exploration of large astronomical data sets.

astro-ph.IM↗

arXiv2020

Photometric Redshifts with the LSST II: The Impact of Near-Infrared and Near-Ultraviolet Photometry

Accurate photometric redshift (photo-$z$) estimates are essential to the cosmological science goals of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST). In this work we use simulated photometry for mock galaxy catalogs to explore how LSST photo-$z$ estimates can be improved by the addition of near-infrared (NIR) and/or ultraviolet (UV) photometry from the Euclid, WFIRST, and/or CASTOR space telescopes. Generally, we find that deeper optical photometry can reduce the standard deviation of the photo-$z$ estimates more than adding NIR or UV filters, but that additional filters are the only way to significantly lower the fraction of galaxies with catastrophically under- or over-estimated photo-$z$. For Euclid, we find that the addition of ${JH}$ $5σ$ photometric detections can reduce the standard deviation for galaxies with $z>1$ ($z>0.3$) by ${\sim}20\%$ (${\sim}10\%$), and the fraction of outliers by ${\sim}40\%$ (${\sim}25\%$). For WFIRST, we show how the addition of deep ${YJHK}$ photometry could reduce the standard deviation by ${\gtrsim}50\%$ at $z>1.5$ and drastically reduce the fraction of outliers to just ${\sim}2\%$ overall. For CASTOR, we find that the addition of its ${UV}$ and $u$-band photometry could reduce the standard deviation by ${\sim}30\%$ and the fraction of outliers by ${\sim}50\%$ for galaxies with $z<0.5$. We also evaluate the photo-$z$ results within sky areas that overlap with both the NIR and UV surveys, and when spectroscopic training sets built from the surveys' small-area deep fields are used.

astro-ph.IM↗

arXiv2020

Applying Information Theory to Design Optimal Filters for Photometric Redshifts

In this paper we apply ideas from information theory to create a method for the design of optimal filters for photometric redshift estimation. We show the method applied to a series of simple example filters in order to motivate an intuition for how photometric redshift estimators respond to the properties of photometric passbands. We then design a realistic set of six filters covering optical wavelengths that optimize photometric redshifts for $z <= 2.3$ and $i < 25.3$. We create a simulated catalog for these optimal filters and use our filters with a photometric redshift estimation code to show that we can improve the standard deviation of the photometric redshift error by 7.1% overall and improve outliers 9.9% over the standard filters proposed for the Large Synoptic Survey Telescope (LSST). We compare features of our optimal filters to LSST and find that the LSST filters incorporate key features for optimal photometric redshift estimation. Finally, we describe how information theory can be applied to a range of optimization problems in astronomy.

astro-ph.IM↗

arXiv2019

Algorithms and Statistical Models for Scientific Discovery in the Petabyte Era

The field of astronomy has arrived at a turning point in terms of size and complexity of both datasets and scientific collaboration. Commensurately, algorithms and statistical models have begun to adapt --- e.g., via the onset of artificial intelligence --- which itself presents new challenges and opportunities for growth. This white paper aims to offer guidance and ideas for how we can evolve our technical and collaborative frameworks to promote efficient algorithmic development and take advantage of opportunities for scientific discovery in the petabyte era. We discuss challenges for discovery in large and complex data sets; challenges and requirements for the next stage of development of statistical methodologies and algorithmic tool sets; how we might change our paradigms of collaboration and education; and the ethical implications of scientists' contributions to widely applicable algorithms and computational modeling. We start with six distinct recommendations that are supported by the commentary following them. This white paper is related to a larger corpus of effort that has taken place within and around the Petabytes to Science Workshops (https://petabytestoscience.github.io/).

astro-ph.IM↗

arXiv2019

AXS: A framework for fast astronomical data processing based on Apache Spark

We introduce AXS (Astronomy eXtensions for Spark), a scalable open-source astronomical data analysis framework built on Apache Spark, a widely used industry-standard engine for big data processing. Building on capabilities present in Spark, AXS aims to enable querying and analyzing almost arbitrarily large astronomical catalogs using familiar Python/AstroPy concepts, DataFrame APIs, and SQL statements. We achieve this by i) adding support to Spark for efficient on-line positional cross-matching and ii) supplying a Python library supporting commonly-used operations for astronomical data analysis. To support scalable cross-matching, we developed a variant of the ZONES algorithm (Gray et al. 2004) capable of operating in distributed, shared-nothing architecture. We couple this to a data partitioning scheme that enables fast catalog cross-matching and handles the data skew often present in deep all-sky data sets. The cross-match and other often-used functionalities are exposed to the end users through an easy-to-use Python API. We demonstrate AXS' technical and scientific performance on SDSS, ZTF, Gaia DR2, and AllWise catalogs. Using AXS we were able to perform on-the-fly cross-match of Gaia DR2 (1.8 billion rows) and AllWise (900 million rows) data sets in ~ 30 seconds. We discuss how cloud-ready distributed systems like AXS provide a natural way to enable comprehensive end-user analyses of large datasets such as LSST.

astro-ph.IM↗

arXiv2019

Enabling Deep All-Sky Searches of Outer Solar System Objects

A foundational goal of the Large Synoptic Survey Telescope (LSST) is to map the Solar System small body populations that provide key windows into understanding of its formation and evolution. This is especially true of the populations of the Outer Solar System -- objects at the orbit of Neptune $r > 30$AU and beyond. In this whitepaper, we propose a minimal change to the LSST cadence that can greatly enhance LSST's ability to discover faint distant Solar System objects across the entire wide-fast-deep (WFD) survey area. Specifically, we propose that the WFD cadence be constrained so as to deliver least one sequence of $\gtrsim 10$ visits per year taken in a $\sim 10$ day period in any combination of $g, r$, and $i$ bands. Combined with advanced shift-and-stack algorithms (Whidden et al. 2019) this modification would enable a nearly complete census of the outer Solar System to $\sim 25.5$ magnitude, yielding $4-8$x more KBO discoveries than with single-epoch baseline, and enabling rapid identification and follow-up of unusual distant Solar System objects in $\gtrsim 5$x greater volume of space. These increases would enhance the science cases discussed in Schwamb et al. (2018) whitepaper, including probing Neptune's past migration history as well as discovering hypothesized planet(s) beyond the orbit of Neptune (or at least placing significant constraints on their existence).

astro-ph.EP↗

arXiv2019

Fast algorithms for slow moving asteroids: constraints on the distribution of Kuiper Belt Objects

We introduce a new computational technique for searching for faint moving sources in astronomical images. Starting from a maximum likelihood estimate for the probability of the detection of a source within a series of images, we develop a massively parallel algorithm for searching through candidate asteroid trajectories that utilizes Graphics Processing Units (GPU). This technique can search over 10^10 possible asteroid trajectories in stacks of the order 10-15 4K x 4K images in under a minute using a single consumer grade GPU. We apply this algorithm to data from the 2015 campaign of the High Cadence Transient Survey (HiTS) obtained with the Dark Energy Camera (DECam). We find 39 previously unknown Kuiper Belt Objects in the 150 square degrees of the survey. Comparing these asteroids to an existing model for the inclination distribution of the Kuiper Belt we demonstrate that we recover a KBO population above our detection limit consistent with previous studies. Software used in this analysis is made available as an open source package.

astro-ph.IM↗