arXiv subjects

Stéfan van der Walt

Publications and source records attributed to Stéfan van der Walt.

6 recordsLinked to original sources

arXiv2023

nbi: the Astronomer's Package for Neural Posterior Estimation

Despite the promise of Neural Posterior Estimation (NPE) methods in astronomy, the adaptation of NPE into the routine inference workflow has been slow. We identify three critical issues: the need for custom featurizer networks tailored to the observed data, the inference inexactness, and the under-specification of physical forward models. To address the first two issues, we introduce a new framework and open-source software nbi (Neural Bayesian Inference), which supports both amortized and sequential NPE. First, nbi provides built-in "featurizer" networks with demonstrated efficacy on sequential data, such as light curve and spectra, thus obviating the need for this customization on the user end. Second, we introduce a modified algorithm SNPE-IS, which facilities asymptotically exact inference by using the surrogate posterior under NPE only as a proposal distribution for importance sampling. These features allow nbi to be applied off-the-shelf to astronomical inference problems involving light curves and spectra. We discuss how nbi may serve as an effective alternative to existing methods such as Nested Sampling. Our package is at https://github.com/kmzzhang/nbi.

astro-ph.IM↗

arXiv2023

A data science platform to enable time-domain astronomy

SkyPortal is an open-source software package designed to efficiently discover interesting transients, manage follow-up, perform characterization, and visualize the results. By enabling fast access to archival and catalog data, cross-matching heterogeneous data streams, and the triggering and monitoring of on-demand observations for further characterization, a SkyPortal-based platform has been operating at scale for 2 yr for the Zwicky Transient Facility Phase II community, with hundreds of users, containing tens of millions of time-domain sources, interacting with dozens of telescopes, and enabling community reporting. While SkyPortal emphasizes rich user experiences (UX) across common frontend workflows, recognizing that scientific inquiry is increasingly performed programmatically, SkyPortal also surfaces an extensive and well-documented API system. From backend and frontend software to data science analysis tools and visualization frameworks, the SkyPortal design emphasizes the re-use and leveraging of best-in-class approaches, with a strong extensibility ethos. For instance, SkyPortal now leverages ChatGPT large-language models (LLMs) to automatically generate and surface source-level human-readable summaries. With the imminent re-start of the next-generation of gravitational wave detectors, SkyPortal now also includes dedicated multi-messenger features addressing the requirements of rapid multi-messenger follow-up: multi-telescope management, team/group organizing interfaces, and cross-matching of multi-messenger data streams with time-domain optical surveys, with interfaces sufficiently intuitive for the newcomers to the field. (abridged)

astro-ph.IM↗

arXiv2022

HEALPix Alchemy: Fast All-Sky Geometry and Image Arithmetic in a Relational Database for Multimessenger Astronomy Brokers

Efficient searches for electromagnetic counterparts to gravitational wave, high-energy neutrino, and gamma-ray burst events demand rapid processing of image arithmetic and geometry set operations in a database to cross-match galaxy catalogs, observation footprints, and all-sky images. Here we introduce HEALPix Alchemy, an open-source, pure Python implementation of a set of methods that enables rapid all-sky geometry calculations. HEALPix Alchemy is built upon HEALPix, a spatial indexing strategy that is widely used in astronomical databases as well as the native format of LIGO-Virgo-KAGRA gravitational-wave sky localization maps. Our approach leverages new multirange types built into the PostgreSQL 14 database engine. This enables fast all-sky queries against probabilistic multimessenger event localizations and telescope survey footprints. Questions such as "What are the galaxies contained within the 90% credible region of an event?" and "What is the rank-ordered list of the fields within an observing footprint with the highest probability of containing the event?" can be performed in less than a few seconds on commodity hardware using off-the-shelf cloud-managed database implementations without server-side database extensions. Common queries scale roughly linearly with the number of telescope pointings. As the number of fields grows into the hundreds or thousands, HEALPix Alchemy is orders of magnitude faster than other implementations. HEALPix Alchemy is now used as the spatial geometry engine within SkyPortal, which forms the basis of the Zwicky Transient Facility transient marshal, called Fritz.

astro-ph.IM↗

arXiv2021

A reusable pipeline for large-scale fiber segmentation on unidirectional fiber beds using fully convolutional neural networks

Fiber-reinforced ceramic-matrix composites are advanced materials resistant to high temperatures, with application to aerospace engineering. Their analysis depends on the detection of embedded fibers, with semi-supervised techniques usually employed to separate fibers within the fiber beds. Here we present an open computational pipeline to detect fibers in ex-situ X-ray computed tomography fiber beds. To separate the fibers in these samples, we tested four different architectures of fully convolutional neural networks. When comparing our neural network approach to a semi-supervised one, we obtained Dice and Matthews coefficients greater than $92.28 \pm 9.65\%$, reaching up to $98.42 \pm 0.03 \%$, showing that the network results are close to the human-supervised ones in these fiber beds, in some cases separating fibers that human-curated algorithms could not find. The software we generated in this project is open source, released under a permissive license, and can be freely adapted and re-used in other domains. All data and instructions on how to download and use it are also available.

eess.IV↗

arXiv2017

A recurrent neural network for classification of unevenly sampled variable stars

Astronomical surveys of celestial sources produce streams of noisy time series measuring flux versus time ("light curves"). Unlike in many other physical domains, however, large (and source-specific) temporal gaps in data arise naturally due to intranight cadence choices as well as diurnal and seasonal constraints. With nightly observations of millions of variable stars and transients from upcoming surveys, efficient and accurate discovery and classification techniques on noisy, irregularly sampled data must be employed with minimal human-in-the-loop involvement. Machine learning for inference tasks on such data traditionally requires the laborious hand-coding of domain-specific numerical summaries of raw data ("features"). Here we present a novel unsupervised autoencoding recurrent neural network (RNN) that makes explicit use of sampling times and known heteroskedastic noise properties. When trained on optical variable star catalogs, this network produces supervised classification models that rival other best-in-class approaches. We find that autoencoded features learned on one time-domain survey perform nearly as well when applied to another survey. These networks can continue to learn from new unlabeled observations and may be used in other unsupervised tasks such as forecasting and anomaly detection.

astro-ph.IM↗

arXiv2016

cesium: Open-Source Platform for Time-Series Inference

Inference on time series data is a common requirement in many scientific disciplines and internet of things (IoT) applications, yet there are few resources available to domain scientists to easily, robustly, and repeatably build such complex inference workflows: traditional statistical models of time series are often too rigid to explain complex time domain behavior, while popular machine learning packages require already-featurized dataset inputs. Moreover, the software engineering tasks required to instantiate the computational platform are daunting. cesium is an end-to-end time series analysis framework, consisting of a Python library as well as a web front-end interface, that allows researchers to featurize raw data and apply modern machine learning techniques in a simple, reproducible, and extensible way. Users can apply out-of-the-box feature engineering workflows as well as save and replay their own analyses. Any steps taken in the front end can also be exported to a Jupyter notebook, so users can iterate between possible models within the front end and then fine-tune their analysis using the additional capabilities of the back-end library. The open-source packages make us of many use modern Python toolkits, including xarray, dask, Celery, Flask, and scikit-learn.

cs.MS↗