arXiv subjects

Kai A. Horstmann

Publications and source records attributed to Kai A. Horstmann.

3 recordsLinked to original sources

arXiv2026

A case study of evaluating AI agents on a neuroscience data-to-discovery pipeline

Agentic AI offers a promising path to automating software development bottlenecks in scientific research pipelines, particularly for stages that take domain experts days to months to build and where correctness and robustness matter more than implementation details. We present an empirical study of general-purpose coding agents on a fly optogenetics data-to-discovery pipeline. We assess agents on tasks and datasets substantially larger than existing benchmarks and evaluation criteria grounded in domain expert standards. We show that agents can solve several pipeline stages, suggesting stage-level automation is tractable. By analyzing agents' code iterations, we show they struggle most without a pre-defined criterion, when they must instead use their scientific judgment to assess their current solution. Mirroring scientists, they sometimes attempt visual inspection of intermediate outputs for self-evaluation, but largely fail to interpret what they see or act on it appropriately. Solving the end-to-end pipeline requires stringing together successes across all stages, which is beyond agents' current abilities. We identify challenges largely absent from existing benchmarks, including computational resource management and generalization to held-out data. Finally, we distill principles for constructing scientific tasks and rigorous evaluation criteria for open-ended problems.

cs.AI↗

arXiv2025

Simple Agents Outperform Experts in Biomedical Imaging Workflow Optimization

Adapting production-level computer vision tools to bespoke scientific datasets is a critical "last mile" bottleneck. Current solutions are impractical: fine-tuning requires large annotated datasets scientists often lack, while manual code adaptation costs scientists weeks to months of effort. We consider using AI agents to automate this manual coding, and focus on the open question of optimal agent design for this targeted task. We introduce a systematic evaluation framework for agentic code optimization and use it to study three production-level biomedical imaging pipelines. We demonstrate that a simple agent framework consistently generates adaptation code that outperforms human-expert solutions. Our analysis reveals that common, complex agent architectures are not universally beneficial, leading to a practical roadmap for agent design. We open source our framework and validate our approach by deploying agent-generated functions into a production pipeline, demonstrating a clear pathway for real-world impact.

cs.CV↗

arXiv2025

Inference-Time Scaling of Diffusion Models for Infrared Data Generation

Infrared imagery enables temperature-based scene understanding using passive sensors, particularly under conditions of low visibility where traditional RGB imaging fails. Yet, developing downstream vision models for infrared applications is hindered by the scarcity of high-quality annotated data, due to the specialized expertise required for infrared annotation. While synthetic infrared image generation has the potential to accelerate model development by providing large-scale, diverse training data, training foundation-level generative diffusion models in the infrared domain has remained elusive due to limited datasets. In light of such data constraints, we explore an inference-time scaling approach using a domain-adapted CLIP-based verifier for enhanced infrared image generation quality. We adapt FLUX.1-dev, a state-of-the-art text-to-image diffusion model, to the infrared domain by finetuning it on a small sample of infrared images using parameter-efficient techniques. The trained verifier is then employed during inference to guide the diffusion sampling process toward higher quality infrared generations that better align with input text prompts. Empirically, we find that our approach leads to consistent improvements in generation quality, reducing FID scores on the KAIST Multispectral Pedestrian Detection Benchmark dataset by 10% compared to unguided baseline samples. Our results suggest that inference-time guidance offers a promising direction for bridging the domain gap in low-data infrared settings.

cs.CV↗