arXiv subjects

Kiri L. Wagstaff

Publications and source records attributed to Kiri L. Wagstaff.

At least 19 recordsLinked to original sources

arXiv2026

AAAI-26 Dual Submissions: Novel Challenges

Dual submissions, in which identical or substantially similar papers are simultaneously submitted to one or more archival venues, without cross-citation or disclosure, are a growing problem for the AAAI Conference and other scientific publication venues. These submissions increase the burden on the peer-review system and pollute the scientific record. As part of the AAAI-26 review process, we (conference organizers) compared AAAI main-track submissions to nine other archival venues with overlapping review periods. We also searched for dual submissions within the AAAI-26 main track. We employed title+abstract similarity assessment to prioritize highly similar paper pairs for subsequent triage by an LLM-based overlap assessment tool, followed by manual review of the highest severity pairs. Manual review of such pairs led to the desk-rejection of 141 AAAI-26 main-track submissions. We seek to alert future organizers, and the broader artificial intelligence research community, to the enormous growth in dual submissions. The incidence of exact duplicate submissions, which are easy to detect, has been eclipsed by the number of papers that use different words to describe the same contribution, which are extremely time-consuming to detect. The growth in this phenomenon is likely facilitated by increasing access to generative AI tools. We include several recommendations for addressing this challenge, including (1) updating the AAAI Multiple Submission Policy and educating the community about acceptable practice, (2) having dual-submission checking tools in place before submissions close, (3) working across venues to converge on consistent policies and penalties to aid in reducing the incidence of dual submission, and (4) creating a community-driven adversarial challenge to accelerate the development of robust detection tools.

cs.DL↗

arXiv2026

Position: Every Ground Truth is a Human Construction, not an Objective Truth

Ground truth datasets play a fundamental role as reference values in the training and evaluation of machine learning models. This position paper argues that ground truths are not neutral objective measurements that are naturally given, but instead that they are constructed by arrangements of humans and technologies. We argue that the ML community will benefit from articulating and discussing these often invisible or unreported choices and acknowledging that reference data sets are contingent, not universal. Focusing on the situated and context-dependent nature of ground truths can improve reliability by enabling a better informed perspective on where, when, and how the datasets, and the models they have shaped, can best be used. We argue for increasing `situated reliability' which includes articulating the limits and strengths of models and their truth claims. Finally, paying more attention to the construction of ground truths can support transparency, accountability, and interdisciplinary work.

cs.LG↗

arXiv2026

Interactive Mars Image Content-Based Search with Interpretable Machine Learning

The NASA Planetary Data System (PDS) hosts millions of images of planets, moons, and other bodies collected throughout many missions. The ever-expanding nature of data and user engagement demands an interpretable content classification system to support scientific discovery and individual curiosity. In this paper, we leverage a prototype-based architecture to enable users to understand and validate the evidence used by a classifier trained on images from the Mars Science Laboratory (MSL) Curiosity rover mission. In addition to providing explanations, we investigate the diversity and correctness of evidence used by the content-based classifier. The work presented in this paper will be deployed on the PDS Image Atlas, replacing its non-interpretable counterpart.

cs.CV↗

arXiv2026

Realistic Handwritten Multi-Digit Writer (MDW) Number Recognition Challenges

Isolated digit classification has served as a motivating problem for decades of machine learning research. In real settings, numbers often occur as multiple digits, all written by the same person. Examples include ZIP Codes, handwritten check amounts, and appointment times. In this work, we leverage knowledge about the writers of NIST digit images to create more realistic benchmark multi-digit writer (MDW) data sets. As expected, we find that classifiers may perform well on isolated digits yet do poorly on multi-digit number recognition. If we want to solve real number recognition problems, additional advances are needed. The MDW benchmarks come with task-specific performance metrics that go beyond typical error calculations to more closely align with real-world impact. They also create opportunities to develop methods that can leverage task-specific knowledge to improve performance well beyond that of individual digit classification methods.

cs.CV↗

arXiv2026

AI-Assisted Peer Review at Scale: The AAAI-26 AI Review Pilot

Scientific peer review faces mounting strain as submission volumes surge, making it increasingly difficult to sustain review quality, consistency, and timeliness. Recent advances in AI have led the community to consider its use in peer review, yet a key unresolved question is whether AI can generate technically sound reviews at real-world conference scale. Here we report the first large-scale field deployment of AI-assisted peer review: every main-track submission at AAAI-26 received one clearly identified AI review from a state-of-the-art system. The system combined frontier models, tool use, and safeguards in a multi-stage process to generate reviews for all 22,977 full-review papers in less than a day. A large-scale survey of AAAI-26 authors and program committee members showed that participants not only found AI reviews useful, but actually preferred them to human reviews on key dimensions such as technical accuracy and research suggestions. We also introduce a novel benchmark and find that our system substantially outperforms a simple LLM-generated review baseline at detecting a variety of scientific weaknesses. Together, these results show that state-of-the-art AI methods can already make meaningful contributions to scientific peer review at conference scale, opening a path toward the next generation of synergistic human-AI teaming for evaluating research.

cs.AI↗

arXiv2024

Evaluating Terrain-Dependent Performance for Martian Frost Detection in Visible Satellite Observations

Seasonal frosting and defrosting on the surface of Mars is hypothesized to drive both climate processes and the formation and evolution of geomorphological features such as gullies. Past studies have focused on manually analyzing the behavior of the frost cycle in the northern mid-latitude region of Mars using high-resolution visible observations from orbit. Extending these studies globally requires automating the detection of frost using data science techniques such as convolutional neural networks. However, visible indications of frost presence can vary significantly depending on the geologic context on which the frost is superimposed. In this study, we (1) present a novel approach for spatially partitioning data to reduce biases in model performance estimation, (2) illustrate how geologic context affects automated frost detection, and (3) propose mitigations to observed biases in automated frost detection.

cs.CV↗

arXiv2023

Cosmic Microwave Background Recovery: A Graph-Based Bayesian Convolutional Network Approach

The cosmic microwave background (CMB) is a significant source of knowledge about the origin and evolution of our universe. However, observations of the CMB are contaminated by foreground emissions, obscuring the CMB signal and reducing its efficacy in constraining cosmological parameters. We employ deep learning as a data-driven approach to CMB cleaning from multi-frequency full-sky maps. In particular, we develop a graph-based Bayesian convolutional neural network based on the U-Net architecture that predicts cleaned CMB with pixel-wise uncertainty estimates. We demonstrate the potential of this technique on realistic simulated data based on the Planck mission. We show that our model accurately recovers the cleaned CMB sky map and resulting angular power spectrum while identifying regions of uncertainty. Finally, we discuss the current challenges and the path forward for deploying our model for CMB recovery on real observations.

cs.LG↗

arXiv2023

Hidden Heterogeneity: When to Choose Similarity-Based Calibration

Trustworthy classifiers are essential to the adoption of machine learning predictions in many real-world settings. The predicted probability of possible outcomes can inform high-stakes decision making, particularly when assessing the expected value of alternative decisions or the risk of bad outcomes. These decisions require well-calibrated probabilities, not just the correct prediction of the most likely class. Black-box classifier calibration methods can improve the reliability of a classifier's output without requiring retraining. However, these methods are unable to detect subpopulations where calibration could also improve prediction accuracy. Such subpopulations are said to exhibit "hidden heterogeneity" (HH), because the original classifier did not detect them. This paper proposes a quantitative measure for HH. It also introduces two similarity-weighted calibration methods that can address HH by adapting locally to each test item: SWC weights the calibration set by similarity to the test item, and SWC-HH explicitly incorporates hidden heterogeneity to filter the calibration set. Experiments show that the improvements in calibration achieved by similarity-based calibration methods correlate with the amount of HH present and, given sufficient calibration data, generally exceed calibration achieved by global methods. HH can therefore serve as a useful diagnostic tool for identifying when local calibration methods would be beneficial.

cs.LG↗

arXiv2022

Visualizing Image Content to Explain Novel Image Discovery

The initial analysis of any large data set can be divided into two phases: (1) the identification of common trends or patterns and (2) the identification of anomalies or outliers that deviate from those trends. We focus on the goal of detecting observations with novel content, which can alert us to artifacts in the data set or, potentially, the discovery of previously unknown phenomena. To aid in interpreting and diagnosing the novel aspect of these selected observations, we recommend the use of novelty detection methods that generate explanations. In the context of large image data sets, these explanations should highlight what aspect of a given image is new (color, shape, texture, content) in a human-comprehensible form. We propose DEMUD-VIS, the first method for providing visual explanations of novel image content by employing a convolutional neural network (CNN) to extract image features, a method that uses reconstruction error to detect novel content, and an up-convolutional network to convert CNN feature representations back into image space. We demonstrate this approach on diverse images from ImageNet, freshwater streams, and the surface of Mars.

cs.LG↗

arXiv2022

Using Machine Learning to Reduce Observational Biases When Detecting New Impacts on Mars

The current inventory of recent (fresh) impacts on Mars shows a strong bias towards areas of low thermal inertia. These areas are generally visually bright, and impacts create dark scours and rays that make them easier to detect. It is expected that impacts occur at a similar rate in areas of higher thermal inertia, but those impacts are under-detected. This study investigates the use of a trained machine learning classifier to increase the detection of fresh impacts on Mars using CTX data. This approach discovered 69 new fresh impacts that have been confirmed with follow-up HiRISE images. We found that examining candidates partitioned by thermal inertia (TI) values, which is only possible due to the large number of machine learning candidates, helps reduce the observational bias and increase the number of known high-TI impacts.

cs.LG↗

arXiv2021

Integrating Novelty Detection Capabilities with MSL Mastcam Operations to Enhance Data Analysis

While innovations in scientific instrumentation have pushed the boundaries of Mars rover mission capabilities, the increase in data complexity has pressured Mars Science Laboratory (MSL) and future Mars rover operations staff to quickly analyze complex data sets to meet progressively shorter tactical and strategic planning timelines. MSLWEB is an internal data tracking tool used by operations staff to perform first pass analysis on MSL image sequences, a series of products taken by the Mast camera, Mastcam. Mastcam's multiband multispectral image sequences require more complex analysis compared to standard 3-band RGB images. Typically, these are analyzed using traditional methods to identify unique features within the sequence. Given the short time frame of tactical planning in which downlinked images might need to be analyzed (within 5-10 hours before the next uplink), there exists a need to triage analysis time to focus on the most important sequences and parts of a sequence. We address this need by creating products for MSLWEB that use novelty detection to help operations staff identify unusual data that might be diagnostic of new or atypical compositions or mineralogies detected within an imaging scene. This was achieved in two ways: 1) by creating products for each sequence to identify novel regions in the image, and 2) by assigning multispectral sequences a sortable novelty score. These new products provide colorized heat maps of inferred novelty that operations staff can use to rapidly review downlinked data and focus their efforts on analyzing potentially new kinds of diagnostic multispectral signatures. This approach has the potential to guide scientists to new discoveries by quickly drawing their attention to often subtle variations not detectable with simple color composites.

cs.LG↗

arXiv2018

Interpretable Discovery in Large Image Data Sets

Automated detection of new, interesting, unusual, or anomalous images within large data sets has great value for applications from surveillance (e.g., airport security) to science (observations that don't fit a given theory can lead to new discoveries). Many image data analysis systems are turning to convolutional neural networks (CNNs) to represent image content due to their success in achieving high classification accuracy rates. However, CNN representations are notoriously difficult for humans to interpret. We describe a new strategy that combines novelty detection with CNN image features to achieve rapid discovery with interpretable explanations of novel image content. We applied this technique to familiar images from ImageNet as well as to a scientific image collection from planetary science.

cs.LG↗

arXiv2016

A Machine Learning Classifier for Fast Radio Burst Detection at the VLBA

Time domain radio astronomy observing campaigns frequently generate large volumes of data. Our goal is to develop automated methods that can identify events of interest buried within the larger data stream. The V-FASTR fast transient system was designed to detect rare fast radio bursts (FRBs) within data collected by the Very Long Baseline Array. The resulting event candidates constitute a significant burden in terms of subsequent human reviewing time. We have trained and deployed a machine learning classifier that marks each candidate detection as a pulse from a known pulsar, an artifact due to radio frequency interference, or a potential new discovery. The classifier maintains high reliability by restricting its predictions to those with at least 90% confidence. We have also implemented several efficiency and usability improvements to the V-FASTR web-based candidate review system. Overall, we found that time spent reviewing decreased and the fraction of interesting candidates increased. The classifier now classifies (and therefore filters) 80-90% of the candidates, with an accuracy greater than 98%, leaving only the 10-20% most promising candidates to be reviewed by humans.

astro-ph.IM↗

arXiv2016

Limits on Fast Radio Bursts from Four Years of the V-FASTR Experiment

The V-FASTR experiment on the Very Long Baseline Array was designed to detect dispersed pulses of milliseconds duration, such as fast radio bursts (FRBs). We use all V-FASTR data through February 2015 to report V-FASTR's upper limits on the rates of FRBs, and compare these with re-derived rates from Parkes FRB detection experiments. V-FASTR's operation at lambda=20 cm allows direct comparison with the 20 cm Parkes rate, and we derive a power-law limit of γ<-0.4 (95% confidence limit) on the index of FRB source counts, N(>S)\propto S^γ. Using the previously measured FRB rate and the unprecedented amount of survey time spent searching for FRBs at a large range of wavelengths (0.3 cm > λ> 90 cm), we also place frequency-dependent limits on the spectral distribution of FRBs. The most constraining frequencies place two-point spectral index limits of α_{20cm}^{4cm} < 5.8 and α_{90cm}^{20cm} > -7.6, where fluence F \propto f^αif we assume true the burst rate reported by Champion et al. (2016) of R(F~0.6 Jy ms) = 7 x 10^3 sky^{-1} day^{-1} (for bursts of ~3 ms duration). This upper limit on αsuggests that if FRBs are extragalactic but non-cosmological, that on average they are not experiencing excessive free-free absorption due to a medium with high optical depth (assuming temperature ~8,000 K), which excessively invert their low-frequency spectrum. This in turn implies that the dispersion of FRBs arises in either or both of the intergalactic medium or the host galaxy, rather than from the source itself.

astro-ph.HE↗

arXiv2013

Comparing the WFC3 IR Grism Stare and Spatial-Scan Observations for Exoplanet Characterization

We report on a detailed study of the measurement stability for WFC3 IR grism stare and spatial scan observations. The excess measurement noise for both modes is established by comparing the observed and theoretical measurement uncertainties. We find that the stare-mode observations produce differential measurements that are consistent and achieve $\sim\,1.3$ times photon-limited measurement precision. In contrast, the spatial-scan mode observations produce measurements which are inconsistent, non-Gaussian, and have higher excess noise corresponding to $\sim\,2$ times the photon-limited precision. The inferior quality of the spatial scan observations is due to spatial-temporal variability in the detector performance which we measure and map. The non-Gaussian nature of spatial scan measurements makes the use of $χ^2$ and the determination of formal confidence intervals problematic and thus renders the comparison of spatial scan data with theoretical models or other measurements difficult. With better measurement stability and no evidence for non-Gaussianity, stare mode observations offer a significant advantage for characterizing transiting exoplanet systems.

astro-ph.IM↗

arXiv2013

Real Time Event Detection in Astronomical Data Streams: Lessons from the VLBA

A new generation of observational science instruments is dramatically increasing collected data volumes in a range of fields. These instruments include the Square Kilometre Array (SKA), Large Synoptic Survey Telescope (LSST), terrestrial sensor networks, and NASA satellites participating in "decadal survey" missions. Their unprecedented coverage and sensitivity will likely reveal wholly new categories of unexpected and transient events. Commensal methods passively analyze these data streams, recognizing anomalous events of scientific interest and reacting in real time. We report on a case example: V-FASTR, an ongoing commensal experiment at the Very Long Baseline Array (VLBA) that uses online adaptive pattern recognition to search for anomalous fast radio transients. V-FASTR triages a millisecond-resolution stream of data and promotes candidate anomalies for further offline analysis. It tunes detection parameters in real time, injecting synthetic events to continually retrain itself for optimum performance. This self-tuning approach retains sensitivity to weak signals while adapting to changing instrument configurations and noise conditions. The system has operated since July 2011, making it the longest-running real time commensal radio transient experiment to date.

astro-ph.IM↗

arXiv2013

A framework for interpreting fast radio transients search experiments: application to the V-FASTR experiment

We define a framework for determining constraints on the detection rate of fast transient events from a population of underlying sources, with a view to incorporating beam shape, frequency effects, scattering effects, and detection efficiency into the metric. We then demonstrate a method for combining independent datasets into a single event rate constraint diagram, using a probabilistic approach to the limits on parameter space. We apply this new framework to present the latest results from the V-FASTR experiment, a commensal fast transients search using the Very Long Baseline Array (VLBA). In the 20 cm band, V-FASTR now has the ability to probe the regions of parameter space of importance for the observed Lorimer and Keane fast radio transient candidates, by combining the information from observations with differing bandwidths, and properly accounting for the source dispersion measure, VLBA antenna beam shape, experiment time sampling, and stochastic nature of events. We then apply the framework to combine the results of the V-FASTR and ATA Fly's Eye experiments, demonstrating their complementarity. Expectations for fast transients experiments for the SKA Phase I dish array are then computed, and the impact of large differential bandwidths is discussed.

astro-ph.IM↗

arXiv2012

Limits on the event rates of fast radio transients from the V-FASTR experiment

We present the first results from the V-FASTR experiment, a commensal search for fast transient radio bursts using the Very Long Baseline Array (VLBA). V-FASTR is unique in that the widely spaced VLBA antennas provide a discriminant against non-astronomical signals and a mechanism for the localization and identification of events that is not possible with single dishes or short baseline interferometers. Thus far V-FASTR has accumulated over 1300 hours of observation time with the VLBA, between 90 cm and 3 mm wavelength (327 MHz - 86 GHz), providing the first limits on fast transient event rates at high radio frequencies (>1.4 GHz). V-FASTR has blindly detected bright individual pulses from seven known pulsars but has not detected any single-pulse events that would indicate high redshift impulsive bursts of radio emission. At 1.4 GHz, V-FASTR puts limits on fast transient event rates comparable with the PALFA survey at the Arecibo telescope, but generally at lower sensitivities, and comparable to the "fly's eye" survey at the Allen Telescope Array, but with less sky coverage. We also illustrate the likely performance of the Phase 1 SKA dish array for an incoherent fast transient search fashioned on V-FASTR.

astro-ph.HE↗