Searcharxiv⌕ Search

arXiv · 2611.12308

Unmasking shortcut solutions of coding agents on partial differential equations

Abstract

Coding agents now write simulation programs, but benchmarks usually judge only whether their output matches a reference. In manufactured-solution tasks the data determine the exact solution, so a matching output need not come from a solver. We introduce PDEViBench, a benchmark of 290 tasks in six equation families that traces each passing program to the computation behind its output and reruns unclear programs on changed data. A Codex agent (GPT-5.2) passed 80% of 1,450 runs, yet about one pass in ten returned the exact solution without solving the equation. Counting only solved passes changed which task features made problems seem hard and reversed the ranking of two prompts. Once the prompt forbade the shortcut, every pass was a numerical solve, and on tasks without a closed-form solution passing programs stayed correct on changed data. Scientific coding benchmarks should check how each answer was computed as well as whether it matches.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Jiale Linghu, Yangshuai Wang. 2026-11-09. Unmasking shortcut solutions of coding agents on partial differential equations. https://searcharxiv.com/abs/2611.12308

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Adaptive mesh refinement for the Landau-Lifshitz-Gilbert equation

We propose a new adaptive algorithm for the approximation of the Landau--Lifshitz--Gilbert equation via a higher-order tangent plane scheme. We show that the adaptive approximation satisfies an energy inequality and demonstrate numerically that the adaptive algorithm outperforms uniform approaches.

math.NA↗

A one-dimensional model of catheter aspiration in blood vessels

Aspiration thrombectomy is a primary endovascular treatment option for acute ischemic stroke caused by large vessel occlusions, in which a catheter is navigated to the occlusion site and suction is applied to remove the thrombus. This work introduces a reduced one-dimensional model for aspiration in blood vessels that accounts for the elasticity of both the vessel wall and the catheter. The inclusion of vessel wall visco-elasticity transforms the governing equation for the flow rate into a parabolic form, enabling an accurate resolution of the diffuse pressure gradients near the catheter tip observed in CFD simulations. The numerical treatment is based on a relaxation of the hyperbolic subsystem that facilitates nodal solvers, enabling efficient coupling between catheterized and uncatheterized vessel segments, including bifurcations and the catheter tip. A splitting strategy ensures that the visco-elastic terms incur only negligible additional computational cost relative to the purely hyperbolic model. The model is benchmarked against three-dimensional CFD simulations and reference data from the literature. Numerical experiments investigating the role of catheter elasticity and suction force on the hemodynamics are presented, and an uncertainty quantification study demonstrates the suitability of the framework for efficient parameter studies. The results indicate that the proposed model resolves blood flow comparably accurate to three-dimensional simulations at a fraction of the computational cost. While further in vitro and in vivo validation is required, the framework represents a promising step toward computationally efficient preoperative planning and uncertainty-aware decision support in aspiration thrombectomy.

math.NA↗

Exploiting the nonzero diagonal pattern in matrix function computations

We consider the task of approximating a matrix function $f(A)$, where $A$ is a matrix in which only a relatively small number of (not necessarily consecutive) sub- and superdiagonals contain nonzero entries. Approximating $f$ by a low-degree polynomial $p$ allows us to obtain sparse approximations to $f(A)$, which one can efficiently work with (while, in general, $f(A)$ is a dense matrix, even when $A$ is sparse). Our approach is based on carefully inspecting the locations where nonzeros can occur in $p(A)$, and identifying the entries in $A$ that influence them. In particular, we illustrate how this approach can be used for efficiently approximating the trace of $f(A)$ and identify how this approach is related to established (stochastic) probing methods for trace estimation. Another application area in which our approach works particularly well is the computation of functions of Toeplitz matrices. Here, studying the sparsity pattern of $p(A)$ allows us to reduce the computation of the whole matrix polynomial to that of a single small-scale submatrix, yielding an algorithm that scales exceptionally well to large problem sizes.

math.NA↗