arXiv subjects

Peter Clark

Publications and source records attributed to Peter Clark.

At least 37 records · Page 2Linked to original sources

arXiv2025

The rate of extreme coronal line emitters in the Baryon Oscillation Spectroscopic Survey LOWZ sample

Extreme coronal line emitters (ECLEs) are a rare class of galaxy that exhibit strong, high-ionization iron coronal emission lines in their spectra. In some cases, these lines are transient and may be the result of tidal disruption event (TDEs). To test this connection, we calculate the rate of variable ECLEs (vECLEs) at redshift $\sim0.3$. We search for ECLEs in the Baryon Oscillation Spectroscopic Survey (BOSS) LOWZ sample and discover two candidate ECLEs. Using follow-up spectra from the Dark Energy Spectroscopic Instrument and Gemini Multi-Object Spectrograph, and mid-infrared observations from the Wide-field Infrared Survey Explorer, we determine that one of these galaxies is a vECLE. Using this galaxy, we calculate the galaxy-normalized vECLE rate at redshift $\sim0.3$ to be $R_\mathrm{G}=1.6~^{+3.8}_{-1.4}\times10^{-6}~\mathrm{galaxy}^{-1}~\mathrm{yr}^{-1}$ and the mass-normalized rate to be $R_\mathrm{M}=7~^{+16}_{-6}\times10^{-18}~\mathrm{M_\odot^{-1}}~\mathrm{yr}^{-1}$. This is then converted to a volumetric rate of $R_\mathrm{V}=1.8~^{+4.5}_{-1.5}\times10^{-9}~\mathrm{Mpc}^{-3}~\mathrm{yr}^{-1}$. Formally, the LOWZ vECLE rates are $2-4$ times lower than the rates calculated from the Sloan Digital Sky Survey Legacy sample at redshift $\sim0.1$. However, given the large uncertainties on both measurements, they are consistent with each other at $1σ$. Both the galaxy-normalized and volumetric rates are one to two orders of magnitude lower than TDE rates from the literature, consistent with vECLEs being caused by $5-20$ per cent of all TDEs.

astro-ph.HE↗

arXiv2025

CodeScientist: End-to-End Semi-Automated Scientific Discovery with Code-based Experimentation

Despite the surge of interest in autonomous scientific discovery (ASD) of software artifacts (e.g., improved ML algorithms), current ASD systems face two key limitations: (1) they largely explore variants of existing codebases or similarly constrained design spaces, and (2) they produce large volumes of research artifacts (such as automatically generated papers and code) that are typically evaluated using conference-style paper review with limited evaluation of code. In this work we introduce CodeScientist, a novel ASD system that frames ideation and experiment construction as a form of genetic search jointly over combinations of research articles and codeblocks defining common actions in a domain (like prompting a language model). We use this paradigm to conduct hundreds of automated experiments on machine-generated ideas broadly in the domain of agents and virtual environments, with the system returning 19 discoveries, 6 of which were judged as being both at least minimally sound and incrementally novel after a multi-faceted evaluation beyond that typically conducted in prior work, including external (conference-style) review, code review, and replication attempts. Moreover, the discoveries span new tasks, agents, metrics, and data, suggesting a qualitative shift from benchmark optimization to broader discoveries.

cs.AI↗

arXiv2025

The La Silla Schmidt Southern Survey

We present the La Silla Schmidt Southern Survey (LS4), a new wide-field, time-domain survey to be conducted with the 1 m ESO Schmidt telescope. The 268 megapixel LS4 camera mosaics 32 2k$\times$4k fully depleted CCDs, providing a $\sim$20 deg$^2$ field of view with $1''$ pixel$^{-1}$ resolution. The LS4 camera will have excellent performance at longer wavelengths: in a standard 45 s exposure the expected 5$σ$ limiting magnitudes in $g$, $i$, $z$ are $\sim$21.5, $\sim$20.9, and $\sim$20.3 mag (AB), respectively. The telescope design requires a novel filter holder that fixes different bandpasses over each quadrant of the detector. Two quadrants will have $i$ band, while the other two will be $g$ and $z$ band and color information will be obtained by dithering targets across the different quadrants. The majority (90%) of the observing time will be used to conduct a public survey that monitors the extragalactic sky at both moderate (3 d) and high (1 d) cadence, as well as focused observations within the Galactic bulge and plane. Alerts from the public survey will be broadcast to the community via established alert brokers. LS4 will run concurrently with the Vera C. Rubin Observatory's Legacy Survey of Space and Time (LSST). The combination of LS4+LSST will enable detailed holistic monitoring of many nearby transients: high-cadence LS4 observations will resolve the initial rise and peak of the light curve while less-frequent but deeper observations by LSST will characterize the years before and after explosion. Here, we summarize the primary science objectives of LS4 including microlensing events in the Galaxy, extragalactic transients, the search for electromagnetic counterparts to multi-messenger events, and cosmology.

astro-ph.IM↗

arXiv2025

Searching for electromagnetic emission in an AGN from the gravitational wave binary black hole merger candidate S230922g

We carried out long-term monitoring of the LIGO/Virgo/KAGRA binary black hole (BBH) merger candidate S230922g in search of electromagnetic emission from the interaction of the merger remnant with an embedding active galactic nuclei (AGN) accretion disk. Using a dataset primarily composed of wide-field imaging from the Dark Energy Camera (DECam) and supplemented by additional photometric and spectroscopic resources, we searched ~ 70% of the sky area probability for transient phenomena, and discovered 6 counterpart candidates. One especially promising candidate - AT 2023aagj - exhibited temporally varying asymmetric components in spectral broad line regions, a feature potentially indicative of an off-center event such as a BBH merger. This represents the first live search and multiwavelength, photometric, and spectroscopic monitoring of a GW BBH optical counterpart candidate in the disk of an AGN.

astro-ph.HE↗

arXiv2024

From Models to Microtheories: Distilling a Model's Topical Knowledge for Grounded Question Answering

Recent reasoning methods (e.g., chain-of-thought, entailment reasoning) help users understand how language models (LMs) answer a single question, but they do little to reveal the LM's overall understanding, or "theory," about the question's topic, making it still hard to trust the model. Our goal is to materialize such theories - here called microtheories (a linguistic analog of logical microtheories) - as a set of sentences encapsulating an LM's core knowledge about a topic. These statements systematically work together to entail answers to a set of questions to both engender trust and improve performance. Our approach is to first populate a knowledge store with (model-generated) sentences that entail answers to training questions and then distill those down to a core microtheory that is concise, general, and non-redundant. We show that, when added to a general corpus (e.g., Wikipedia), microtheories can supply critical, topical information not necessarily present in the corpus, improving both a model's ability to ground its answers to verifiable knowledge (i.e., show how answers are systematically entailed by documents in the corpus, fully grounding up to +8% more answers), and the accuracy of those grounded answers (up to +8% absolute). We also show that, in a human evaluation in the medical domain, our distilled microtheories contain a significantly higher concentration of topically critical facts than the non-distilled knowledge store. Finally, we show we can quantify the coverage of a microtheory for a topic (characterized by a dataset) using a notion of $p$-relevance. Together, these suggest that microtheories are an efficient distillation of an LM's topic-relevant knowledge, that they can usefully augment existing corpora, and can provide both performance gains and an interpretable, verifiable window into the model's knowledge of a topic.

cs.CL↗

arXiv2024

Learning to Reason via Program Generation, Emulation, and Search

Program synthesis with language models (LMs) has unlocked a large set of reasoning abilities; code-tuned LMs have proven adept at generating programs that solve a wide variety of algorithmic symbolic manipulation tasks (e.g. word concatenation). However, not all reasoning tasks are easily expressible as code, e.g. tasks involving commonsense reasoning, moral decision-making, and sarcasm understanding. Our goal is to extend an LM's program synthesis skills to such tasks and evaluate the results via pseudo-programs, namely Python programs where some leaf function calls are left undefined. To that end, we propose, Code Generation and Emulated EXecution (CoGEX). CoGEX works by (1) training LMs to generate pseudo-programs, (2) teaching them to emulate their generated program's execution, including those leaf functions, allowing the LM's knowledge to fill in the execution gaps; and (3) using them to search over many programs to find an optimal one. To adapt the CoGEX model to a new task, we introduce a method for performing program search to find a single program whose pseudo-execution yields optimal performance when applied to all the instances of a given dataset. We show that our approach yields large improvements compared to standard in-context learning approaches on a battery of tasks, both algorithmic and soft reasoning. This result thus demonstrates that code synthesis can be applied to a much broader class of problems than previously considered. Our released dataset, fine-tuned models, and implementation can be found at \url{https://github.com/nweir127/CoGEX}.

cs.CL↗

arXiv2024

The rate of extreme coronal line emitting galaxies in the Sloan Digital Sky Survey and their relation to tidal disruption events

High-ionization iron coronal lines (CLs) are a rare phenomenon observed in galaxy and quasi-stellar object spectra that are thought to be created by high-energy emission from active galactic nuclei and certain types of transients. In cases known as extreme coronal line emitting galaxies (ECLEs), these CLs are strong and fade away on a timescale of years. The most likely progenitors of these variable CLs are tidal disruption events (TDEs), which produce sufficient high-energy emission to create and sustain the CLs over these timescales. To test the possible connection between ECLEs and TDEs, we present the most complete variable ECLE rate calculation to date and compare the results to TDE rates from the literature. To achieve this, we search for ECLEs in the Sloan Digital Sky Survey (SDSS). We detect sufficiently strong CLs in 16 galaxies, more than doubling the number previously found in SDSS. We find that none of the nine new ECLEs evolve in a manner consistent with that of the five previously discovered variable ECLEs. Using this sample of five variable ECLEs, we calculate the galaxy-normalized rate of variable ECLEs in SDSS to be $R_\mathrm{G}=3.6~^{+2.6}_{-1.8}~(\mathrm{statistical})~^{+5.1}_{-0.0} (\mathrm{systematic})\times10^{-6}~\mathrm{galaxy}^{-1}~\mathrm{yr}^{-1}$. The mass-normalised rate is $R_\mathrm{M}=3.1~^{+2.3}_{-1.5}~(\mathrm{statistical})~^{+4.4}_{-0.0}~(\mathrm{systematic})\times10^{-17}~\mathrm{M_\odot^{-1}}~\mathrm{yr}^{-1}$ and the volumetric rate is $R_\mathrm{V}=7~^{+20}_{-5}~(\mathrm{statistical})~^{+10}_{-0.0}~(\mathrm{systematic})\times10^{-9}~\mathrm{Mpc}^{-3}~\mathrm{yr}^{-1}$. Our rates are one to two orders of magnitude lower than TDE rates from the literature, which suggests that only 10 to 40 per cent of all TDEs produce variable ECLEs. Additional uncertainties in the rates arising from the structure of the interstellar medium have yet to be included.

astro-ph.HE↗

arXiv2024

DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents

Automated scientific discovery promises to accelerate progress across scientific domains. However, developing and evaluating an AI agent's capacity for end-to-end scientific reasoning is challenging as running real-world experiments is often prohibitively expensive or infeasible. In this work we introduce DISCOVERYWORLD, the first virtual environment for developing and benchmarking an agent's ability to perform complete cycles of novel scientific discovery. DISCOVERYWORLD contains a variety of different challenges, covering topics as diverse as radioisotope dating, rocket science, and proteomics, to encourage development of general discovery skills rather than task-specific solutions. DISCOVERYWORLD itself is an inexpensive, simulated, text-based environment (with optional 2D visual overlay). It includes 120 different challenge tasks, spanning eight topics each with three levels of difficulty and several parametric variations. Each task requires an agent to form hypotheses, design and run experiments, analyze results, and act on conclusions. DISCOVERYWORLD further provides three automatic metrics for evaluating performance, based on (a) task completion, (b) task-relevant actions taken, and (c) the discovered explanatory knowledge. We find that strong baseline agents, that perform well in prior published environments, struggle on most DISCOVERYWORLD tasks, suggesting that DISCOVERYWORLD captures some of the novel challenges of discovery, and thus that DISCOVERYWORLD may help accelerate near-term development and assessment of scientific discovery competency in agents. Code available at: www.github.com/allenai/discoveryworld

cs.AI↗

arXiv2024

SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories

Given that Large Language Models (LLMs) have made significant progress in writing code, can they now be used to autonomously reproduce results from research repositories? Such a capability would be a boon to the research community, helping researchers validate, understand, and extend prior work. To advance towards this goal, we introduce SUPER, the first benchmark designed to evaluate the capability of LLMs in setting up and executing tasks from research repositories. SUPERaims to capture the realistic challenges faced by researchers working with Machine Learning (ML) and Natural Language Processing (NLP) research repositories. Our benchmark comprises three distinct problem sets: 45 end-to-end problems with annotated expert solutions, 152 sub problems derived from the expert set that focus on specific challenges (e.g., configuring a trainer), and 602 automatically generated problems for larger-scale development. We introduce various evaluation measures to assess both task success and progress, utilizing gold solutions when available or approximations otherwise. We show that state-of-the-art approaches struggle to solve these problems with the best model (GPT-4o) solving only 16.3% of the end-to-end set, and 46.1% of the scenarios. This illustrates the challenge of this task, and suggests that SUPER can serve as a valuable resource for the community to make and measure progress.

cs.AI↗

arXiv2024

NELLIE: A Neuro-Symbolic Inference Engine for Grounded, Compositional, and Explainable Reasoning

Our goal is a modern approach to answering questions via systematic reasoning where answers are supported by human interpretable proof trees grounded in an NL corpus of authoritative facts. Such a system would help alleviate the challenges of interpretability and hallucination with modern LMs, and the lack of grounding of current explanation methods (e.g., Chain-of-Thought). This paper proposes a new take on Prolog-based inference engines, where we replace handcrafted rules with a combination of neural language modeling, guided generation, and semiparametric dense retrieval. Our implementation, NELLIE, is the first system to demonstrate fully interpretable, end-to-end grounded QA as entailment tree proof search, going beyond earlier work explaining known-to-be-true facts from text. In experiments, NELLIE outperforms a similar-sized state-of-the-art reasoner [Tafjord et al., 2022] while producing knowledge-grounded explanations. We also find NELLIE can exploit both semi-structured and NL text corpora to guide reasoning. Together these suggest a new way to jointly reap the benefits of both modern neural methods and traditional symbolic reasoning.

cs.CL↗

arXiv2024

Enhancing Systematic Decompositional Natural Language Inference Using Informal Logic

Recent language models enable new opportunities for structured reasoning with text, such as the construction of intuitive, proof-like textual entailment trees without relying on brittle formal logic. However, progress in this direction has been hampered by a long-standing lack of a clear protocol for determining what valid compositional entailment is. This absence causes noisy datasets and limited performance gains by modern neuro-symbolic engines. To address these problems, we formulate a consistent and theoretically grounded approach to annotating decompositional entailment and evaluate its impact on LLM-based textual inference. We find that our new dataset, RDTE (Recognizing Decompositional Textual Entailment), has a substantially higher internal consistency (+9%) than prior decompositional entailment datasets. We also find that training an RDTE-oriented entailment classifier via knowledge distillation and employing it in an entailment tree reasoning engine significantly improves both accuracy and proof quality, illustrating the practical benefit of this advance for textual inference.

cs.CL↗

arXiv2024

Digital Socrates: Evaluating LLMs through Explanation Critiques

While LLMs can provide reasoned explanations along with their answers, the nature and quality of those explanations are still poorly understood. In response, our goal is to define a detailed way of characterizing the explanation capabilities of modern models and to create a nuanced, interpretable explanation evaluation tool that can generate such characterizations automatically, without relying on expensive API calls or human annotations. Our approach is to (a) define the new task of explanation critiquing - identifying and categorizing any main flaw in an explanation and providing suggestions to address the flaw, (b) create a sizeable, human-verified dataset for this task, and (c) train an open-source, automatic critique model (called Digital Socrates) using this data. Through quantitative and qualitative analysis, we demonstrate how Digital Socrates is useful for revealing insights about student models by examining their reasoning chains, and how it can provide high-quality, nuanced, automatic evaluation of those model explanations for the first time. Digital Socrates thus fills an important gap in evaluation tools for understanding and improving the explanation behavior of models.

cs.CL↗

arXiv2024

PDDLEGO: Iterative Planning in Textual Environments

Planning in textual environments have been shown to be a long-standing challenge even for current models. A recent, promising line of work uses LLMs to generate a formal representation of the environment that can be solved by a symbolic planner. However, existing methods rely on a fully-observed environment where all entity states are initially known, so a one-off representation can be constructed, leading to a complete plan. In contrast, we tackle partially-observed environments where there is initially no sufficient information to plan for the end-goal. We propose PDDLEGO that iteratively construct a planning representation that can lead to a partial plan for a given sub-goal. By accomplishing the sub-goal, more information is acquired to augment the representation, eventually achieving the end-goal. We show that plans produced by few-shot PDDLEGO are 43% more efficient than generating plans end-to-end on the Coin Collector simulation, with strong performance (98%) on the more complex Cooking World simulation where end-to-end LLMs fail to generate coherent plans (4%).

cs.CL↗

arXiv2024

Light-Curve Structure and Halpha Line Formation in the Tidal Disruption Event AT 2019azh

AT 2019azh is a H+He tidal disruption event (TDE) with one of the most extensive ultraviolet and optical data sets available to date. We present our photometric and spectroscopic observations of this event starting several weeks before and out to approximately two years after the g-band peak brightness and combine them with public photometric data. This extensive data set robustly reveals a change in the light-curve slope and a possible bump in the rising light curve of a TDE for the first time, which may indicate more than one dominant emission mechanism contributing to the pre-peak light curve. Indeed, we find that the MOSFiT-derived parameters of AT 2019azh, which assume reprocessed accretion as the sole source of emission, are not entirely self-consistent. We further confirm the relation seen in previous TDEs whereby the redder emission peaks later than the bluer emission. The post-peak bolometric light curve of AT 2019azh is better described by an exponential decline than by the canonical t^{-5/3} (and in fact any) power-law decline. We find a possible mid-infrared excess around the peak optical luminosity, but cannot determine its origin. In addition, we provide the earliest measurements of the Halpha emission-line evolution and find no significant time delay between the peak of the V-band light curve and that of the Halpha luminosity. These results can be used to constrain future models of TDE line formation and emission mechanisms in general. More pre-peak 1-2 days cadence observations of TDEs are required to determine whether the characteristics observed here are common among TDEs. More importantly, detailed emission models are needed to fully exploit such observations for understanding the emission physics of TDEs.

astro-ph.HE↗

arXiv2024

PROC2PDDL: Open-Domain Planning Representations from Texts

Planning in a text-based environment continues to be a major challenge for AI systems. Recent approaches have used language models to predict a planning domain definition (e.g., PDDL) but have only been evaluated in closed-domain simulated environments. To address this, we present Proc2PDDL , the first dataset containing open-domain procedural texts paired with expert-annotated PDDL representations. Using this dataset, we evaluate state-of-the-art models on defining the preconditions and effects of actions. We show that Proc2PDDL is highly challenging, with GPT-3.5's success rate close to 0% and GPT-4's around 35%. Our analysis shows both syntactic and semantic errors, indicating LMs' deficiency in both generating domain-specific prgorams and reasoning about events. We hope this analysis and dataset helps future progress towards integrating the best of LMs and formal planning.

cs.CL↗

arXiv2024

DiscoveryBench: Towards Data-Driven Discovery with Large Language Models

Can the rapid advances in code generation, function calling, and data analysis using large language models (LLMs) help automate the search and verification of hypotheses purely from a set of provided datasets? To evaluate this question, we present DiscoveryBench, the first comprehensive benchmark that formalizes the multi-step process of data-driven discovery. The benchmark is designed to systematically assess current model capabilities in discovery tasks and provide a useful resource for improving them. Our benchmark contains 264 tasks collected across 6 diverse domains, such as sociology and engineering, by manually deriving discovery workflows from published papers to approximate the real-world challenges faced by researchers, where each task is defined by a dataset, its metadata, and a discovery goal in natural language. We additionally provide 903 synthetic tasks to conduct controlled evaluations across task complexity. Furthermore, our structured formalism of data-driven discovery enables a facet-based evaluation that provides useful insights into different failure modes. We evaluate several popular LLM-based reasoning frameworks using both open and closed LLMs as baselines on DiscoveryBench and find that even the best system scores only 25%. Our benchmark, thus, illustrates the challenges in autonomous data-driven discovery and serves as a valuable resource for the community to make progress.

cs.CL↗

arXiv2024

Skill Set Optimization: Reinforcing Language Model Behavior via Transferable Skills

Large language models (LLMs) have recently been used for sequential decision making in interactive environments. However, leveraging environment reward signals for continual LLM actor improvement is not straightforward. We propose Skill Set Optimization (SSO) for improving LLM actor performance through constructing and refining sets of transferable skills. SSO constructs skills by extracting common subtrajectories with high rewards and generating subgoals and instructions to represent each skill. These skills are provided to the LLM actor in-context to reinforce behaviors with high rewards. Then, SSO further refines the skill set by pruning skills that do not continue to result in high rewards. We evaluate our method in the classic videogame NetHack and the text environment ScienceWorld to demonstrate SSO's ability to optimize a set of skills and perform in-context policy improvement. SSO outperforms baselines by 40% in our custom NetHack task and outperforms the previous state-of-the-art in ScienceWorld by 35%.

cs.LG↗

arXiv2024

Can Language Models Serve as Text-Based World Simulators?

Virtual environments play a key role in benchmarking advances in complex planning and decision-making tasks but are expensive and complicated to build by hand. Can current language models themselves serve as world simulators, correctly predicting how actions change different world states, thus bypassing the need for extensive manual coding? Our goal is to answer this question in the context of text-based simulators. Our approach is to build and use a new benchmark, called ByteSized32-State-Prediction, containing a dataset of text game state transitions and accompanying game tasks. We use this to directly quantify, for the first time, how well LLMs can serve as text-based world simulators. We test GPT-4 on this dataset and find that, despite its impressive performance, it is still an unreliable world simulator without further innovations. This work thus contributes both new insights into current LLM's capabilities and weaknesses, as well as a novel benchmark to track future progress as new models appear.

cs.CL↗