Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.29500

Auditing LLM Alpha-Factor Mining: Metric Provenance and Full-Cohort Reanalysis

Yufeng He
View original source ↗Read source document ↗Export metadata ↓

Abstract

Reliable evaluation of LLM-based alpha mining requires more than positive backtest summaries. We audit a preserved project at three levels: historical run counters, outcome retention, and a separately scoped public reanalysis. Three historical runs report 67, 20, and 11 keep decisions, but the comparison mixes metric populations: a mean over kept candidates is compared with a mean of per-round maxima, and successful evaluations with all returned outcomes. These artifacts cannot identify an architecture advantage. A released 1,837-expression catalog also excludes non-positive OOS outcomes, making its zero-reversal rate non-informative. We then account for all 1,047 unique expressions in a frozen, IS-screened preserved pool using forty A-shares and 2024-2025 rank associations. A versioned public-operator correction yields 923 scorable records, 99 with insufficient valid dates, and 25 unsupported, without modifying the other 990 prior diagnostics. Among scorable records, 671 have positive and 252 negative signed mean rank-IC. Simple untrained references also show positive associations, so this distribution does not establish incremental agent skill or profitability. The case study motivates explicit metric grains, immutable cohort accounting, and provenance-aware interpretation rather than architectural rankings from incomplete records.

Explore related subjects

cs.CE

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Yufeng He. 2026-10-21. Auditing LLM Alpha-Factor Mining: Metric Provenance and Full-Cohort Reanalysis. https://arxiv.org/abs/2610.29500

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Scalable Active Metamaterials for Shape-Morphing

Shape-morphing materials allow structures such as morphing wings, soft robots, and deployable devices to change their shape on demand without mechanical joints or motors. Aperiodic active metamaterials, in which every unit is designed individually, can realize complex programmed shape changes, but their design by structural optimization requires repeated analyses of the full structure, at a cost that grows rapidly with the number of units. This paper presents a scalable active metamaterial (SAM) framework that designs such structures without simulating the assembled structure. The design domain is discretized into unit cells composed of a stiff frame with compliant hinges and a soft, thermally active infill. The stiff frame limits the interaction between neighboring unit cells, which approximately decouples the design problem into two scales. At the macroscale, the deformation required for each unit cell is obtained by a constrained Laplacian mesh-editing formulation with soft data-driven feasibility constraints. At the microscale, the infill geometry of every unit cell is generated by a conditional diffusion model trained on finite-strain finite element simulations of unit cells. All designs are verified by geometrically nonlinear finite element simulations of the assembled structures. The design cost grows approximately linearly with the number of unit cells. The framework is demonstrated on single-point actuation, multi-point control, and shape matching tasks. We further analyze the sources of the remaining error and discuss possible ways to mitigate it. This work establishes a foundation for the systematic, scalable design of highly complex shape-morphing active metamaterials and for understanding the mechanisms governing their target-matching accuracy.

cs.CE↗
arXiv2026

Pushing Biomolecular Utility-Diversity Frontiers with Supergroup Relative Policy Optimization

Biomolecular generators are often adapted with reward feedback to improve task-specific utility, but pushing utility alone can concentrate generation on a narrow family of candidates. Maintaining diversity is difficult because sample diversity is a set-level property. We introduce Supergroup Relative Policy Optimization (SGRPO), a flexible GRPO-style framework that directly constructs rewards from set-level diversity. For each condition, SGRPO samples a supergroup of candidate sets, compares their diversity under the same condition, and redistributes the group diversity reward to individual rollouts through leave-one-out diversity contributions before combining it with rollout-level utility. This design decouples SGRPO from a particular generator, utility reward, or diversity metric, and allows instantiation with different GRPO-style approaches. We evaluate SGRPO on three main settings: de novo small-molecule design, pocket-based small-molecule design, and de novo protein design, instantiating it with both GRPO and Coupled-GRPO across autoregressive and discrete diffusion generators. Across decoding sweeps, SGRPO expands the utility-diversity Pareto frontier and achieves the best frontier-level metrics relative to baselines. Our analyses further show that direct set-level diversity rewards remain effective with small groups and help preserve broader generation-distribution coverage during post-training. The code is available at https://github.com/IDEA-XL/SGRPO.

cs.CE↗
arXiv2026

Tracekit: Tamper-Evident Intent-Reasoning-Action Auditing for AI Agents

AI agents run shell commands, edit repositories, send email and move money with their operators' credentials, yet the record of what they did is usually a log the agent, or whoever runs it, can edit. We present Tracekit (open source, MIT, version 1.4.1), which turns each agent run into evidence a third party can check offline. A signer that the agent cannot reach outside dev mode decides every tool call against its own policy before it runs, and signs the call, decision and result as Ed25519 records in a per-run hash chain inside an RFC 6962 Merkle tree whose checkpoints independent witnesses can cosign. A run exports as one .tkb bundle that two independent verifiers (Python, and TypeScript, also as a browser page) check against keys the verifier pins, reporting integrity and assurance separately. Disagreements between capture layers become signed records. Tracekit also records the person's request and what the model said before each call, holds calls that break a limit the request stated, compares what was asked, said and done, and finds which input caused an action by replay. Of 21 experiments: a signed log alone detects 6 of 8 tamper classes and a witness the other two; every insider attack in two suites is caught; the signer round trip is 0.45 ms median; no acknowledged record was lost in 60 kill -9 rounds; reconciliation flags 12 of 12 seeded discrepancies; the verifiers reject all 33 forgeries. Policy rules are a tripwire, not the boundary: they stop 570 of 570 harmful calls in our corpus but 211 of 420 (50.2%) written blind to them; the protection is a signer and credentials the agent cannot reach. On a held-out pool, a failed test claimed as passing is found in 86% of sessions with 5 false findings in 1,812 honest calls, and replay names the planted cause in 6 of 6 confirmed harmful runs with a 7B model.

cs.CE↗