arXiv subjects

Jacob Levine

Publications and source records attributed to Jacob Levine.

4 recordsLinked to original sources

arXiv2026

Automated Grading of Handwritten Mathematics Using Vision-Capable LLMs

Automated grading systems have enabled scalable assessment for many response types, but handwritten mathematics remains a barrier due to the complexity of multi-step solutions. Vision-capable large language models (LLMs) offer new opportunities here, yet their reliability in authentic instructional settings remains poorly understood. We present an empirical evaluation of an LLM-based grader for handwritten mathematical work using instructor-defined rubrics. Extending a prior pipeline for typed responses, we integrate transcription and rubric-based evaluation of photographic submissions within a single LLM call, evaluating on student work from two university STEM courses. Comparing AI grading decisions against human-assigned ground truth at the rubric-item level, we observe high overall accuracy, with most errors -- 87\% in the best model -- attributable to transcription failures rather than rubric misapplication. We categorize common error modes, including image quality issues, hallucinated content, and incorrect handling of equivalent expressions. These findings highlight both the promise and limitations of LLM-based grading for handwritten mathematics, providing guidance for system design, prompt refinement, and deployment in educational settings.

cs.CY↗

arXiv2026

The NIRISS PASSAGE Spectroscopic Redshift Catalog in COSMOS

We present the Parallel Application of Slitless Spectroscopy to Analyze Galaxy Evolution (PASSAGE) spectroscopic redshift catalog in the COSMOS field. PASSAGE is a JWST Cycle 1 Near Infrared Imager and Slitless Spectrograph (NIRISS) wide-field slitless spectroscopy (WFSS) pure-parallel survey, obtaining near-infrared spectra of thousands of extragalactic sources. 15 out of 63 PASSAGE fields fall within the Hubble Space Telescope (HST) COSMOS footprint, of which 11 overlap with COSMOS-Web, a JWST treasury survey providing additional space-based photometry. We present our custom line-finding algorithm and visual inspection effort used to identify emission lines and derive the spectroscopic redshifts for line-emitting sources in PASSAGE. The line-finding algorithm identifies between ~200 and 950 line-emitting candidates per field, of which typically 47% were identified as true emission lines post visual inspection. We identify 2183 emission line sources at 0.08 5), multi-line emitting sources. This agreement weakens for PASSAGE single-line emitters with ambiguous identities. These single-line emitters are likely mis-identified around 18% of the time based on comparisons to photometric redshifts. We derive stellar masses using PASSAGE photometry and spectroscopic redshifts, in broad agreement with existing COSMOS-Web stellar masses, but with some discrepancy driven by redshift disagreements. We publicly release this spectroscopic redshift catalog, which will enable community-led science in prime extragalactic fields and serve as a crucial dataset for validating Euclid and Roman spectroscopy.

astro-ph.GA↗

arXiv2025

Parallel Application of Slitless Spectroscopy to Analyze Galaxy Evolution (PASSAGE): Survey Overview

During the second half of Cycle 1 of the James Webb Space Telescope (JWST), we conducted the Parallel Application of Slitless Spectroscopy to Analyze Galaxy Evolution (PASSAGE) program. PASSAGE received the largest allocation of JWST observing time in Cycle 1, 591 hours of NIRISS observations to obtain direct near-IR imaging and slitless spectroscopy. About two thirds of these were ultimately executed, to observe 63 high-latitude fields in Pure Parallel mode. These have provided more than ten thousand near-infrared grism spectrograms of faint galaxies. PASSAGE brings unique advantages in studying galaxy evolution: A) Unbiased spectroscopic search, without prior photometric pre-selection. By including the most numerous galaxies, with low masses and strong emission lines, slitless spectroscopy is the indispensable complement to any pre-targeted spectroscopy; B) The combination of several dozen independent fields to overcome cosmic variance; C) Near-infrared spectral coverage, often spanning the full range from 1.0--2.3 $\mu$m, with minimal wavelength gaps, to measure multiple diagnostic rest-frame optical lines, minimizing sensitivity to dust reddening; D) JWST's unprecedented spatial resolution, in some cases using two orthogonal grism orientations, to overcome contamination due to blending of overlapping spectra; E) Discovery of rare bright objects especially for detailed JWST followup. PASSAGE data are public immediately, and our team plans to deliver fully-processed high-level data products. In this PASSAGE overview, we describe the survey and data quality, and present examples of these accomplishments in several areas of current interest in the evolution of emission-line galaxy properties, particularly at low masses.

astro-ph.GA↗

arXiv2025

Contextualizing biological perturbation experiments through language

High-content perturbation experiments allow scientists to probe biomolecular systems at unprecedented resolution, but experimental and analysis costs pose significant barriers to widespread adoption. Machine learning has the potential to guide efficient exploration of the perturbation space and extract novel insights from these data. However, current approaches neglect the semantic richness of the relevant biology, and their objectives are misaligned with downstream biological analyses. In this paper, we hypothesize that large language models (LLMs) present a natural medium for representing complex biological relationships and rationalizing experimental outcomes. We propose PerturbQA, a benchmark for structured reasoning over perturbation experiments. Unlike current benchmarks that primarily interrogate existing knowledge, PerturbQA is inspired by open problems in perturbation modeling: prediction of differential expression and change of direction for unseen perturbations, and gene set enrichment. We evaluate state-of-the-art machine learning and statistical approaches for modeling perturbations, as well as standard LLM reasoning strategies, and we find that current methods perform poorly on PerturbQA. As a proof of feasibility, we introduce Summer (SUMMarize, retrievE, and answeR, a simple, domain-informed LLM framework that matches or exceeds the current state-of-the-art. Our code and data are publicly available at https://github.com/genentech/PerturbQA.

cs.AI↗