arXiv subjects

R. P. Nathan

Publications and source records attributed to R. P. Nathan.

3 recordsLinked to original sources

arXiv2026

Quality Assessment of Spectroscopic Data Reduction Pipelines Using Artificial Intelligence: Scrutinizing Data Release 2 from the DESI Survey

Large spectroscopic surveys now collect data at a scale that makes traditional visual inspection impractical. We present an unsupervised pipeline for spectroscopic quality assessment that requires no labeled training data. The method combines Uniform Manifold Approximation and Projection for dimensionality reduction with Friends-of-Friends clustering to isolate anomalous spectra for targeted review. We apply this pipeline to 58,291,334 spectra across 14,199 tiles from DESI Data Release 2, processing each tile independently to produce a tile-level outlier catalog. In each tile, the pipeline separates a dense core of typical spectra from small, isolated components and singletons, yielding a total of 1,095,816 outlier candidates. The mean tile-level outlier fraction is about 1.96 percent overall, with values of 0.76 percent and 2.36 percent for the dark and bright main-survey programs, respectively. From the visual inspection of 391 outlier candidates from the dark and bright programs of the main survey, we find that 66.8 percent exhibit identifiable spectral anomalies consistent with known reduction and calibration effects. By contrast, only 4.1 percent carry a non-zero quality flag from the standard reduction pipeline. This shows that the method provides a complementary quality-assessment layer to existing pipeline diagnostics and recovers a substantial population of problematic spectra that standard diagnostics miss. Extrapolating to the main-survey catalog, we estimate that approximately 218,000 candidate outliers are free of identifiable reduction artifacts and may correspond to genuine atypical spectra in the context of DESI. The pipeline is scalable, reproducible, and directly comparable across successive data releases, making it a practical quality-assurance monitor for DESI and future multi-object spectroscopic surveys.

astro-ph.IM↗

arXiv2025

Identifying Anomalous DESI Galaxy Spectra with a Variational Autoencoder

The tens of millions of spectra being captured by the Dark Energy Spectroscopic Instrument (DESI) provide tremendous discovery potential. In this work we show how Machine Learning, in particular Variational Autoencoders (VAE), can detect anomalies in a sample of approximately 200,000 DESI spectra comprising galaxies, quasars and stars. We demonstrate that the VAE can compress the dimensionality of a spectrum by a factor of 100, while still retaining enough information to accurately reconstruct spectral features. We then detect anomalous spectra as those with high reconstruction error and those which are isolated in the VAE latent representation. The anomalies identified fall into two categories: spectra with artefacts and spectra with unique physical features. Awareness of the former can help to improve the DESI spectroscopic pipeline; whilst the latter can lead to the identification of new and unusual objects. To further curate the list of outliers, we use the Astronomaly package which employs Active Learning to provide personalised outlier recommendations for visual inspection. In this work we also explore the VAE latent space, finding that different object classes and subclasses are separated despite being unlabelled. We demonstrate the interpretability of this latent space by identifying tracks within it that correspond to various spectral characteristics. For example, we find tracks that correspond to increasing star formation and increase in broad emission lines along the Balmer series. In upcoming work we hope to apply the methods presented here to search for both systematics and astrophysically interesting objects in much larger datasets of DESI spectra.

astro-ph.IM↗

arXiv2025

Finding Pegasus: Enhancing Unsupervised Anomaly Detection in High-Dimensional Data using a Manifold-Based Approach

Unsupervised machine learning methods are well suited to searching for anomalies at scale but can struggle with the high-dimensional representation of many modern datasets, hence dimensionality reduction (DR) is often performed first. In this paper we analyse unsupervised anomaly detection (AD) from the perspective of the manifold created in DR. We present an idealised illustration, "Finding Pegasus", and a novel formal framework with which we categorise AD methods and their results into "on manifold" and "off manifold". We define these terms and show how they differ. We then use this insight to develop an approach of combining AD methods which significantly boosts AD recall without sacrificing precision in situations employing high DR. When tested on MNIST data, our approach of combining AD methods improves recall by as much as 16 percent compared with simply combining with the best standalone AD method (Isolation Forest), a result which shows great promise for its application to real-world data.

cs.LG↗