Searcharxiv⌕ Search

arXiv subjects

Eray Gençay

Publications and source records attributed to Eray Gençay.

2 recordsLinked to original sources

Can a search hold its own bar? In-loop deflation and its exploits in autonomous strategy discovery

Corrections for backtest overfitting such as the deflated Sharpe ratio are applied after a search ends: the searcher maximises raw performance and a separate procedure deflates the reported winner. Autonomous searches driven by large language models perform many implicit backtests before reporting, so this paper asks what changes when the correction moves inside the loop and the acceptance threshold is indexed to the running count and dispersion of the search's own trials. The threshold then becomes an object the search can optimise against: we show analytically that a naive in-loop bar admits a variance-dilution effect, in which mediocre near-duplicate candidates lower the bar because the ledger's dispersion falls faster than its trial-count factor grows. We propose two defences, a pre-registered variance floor and a monotone high-water-mark bar, and evaluate the construction under a protocol frozen before any run, with every in-loop decision replaying exactly from its recorded trace. Across 53 audited hundred-candidate searches over two frontier models, two universes, and two feature spaces, the gated search reports "no certifiable discovery" on 36 of 37 runs, un-gated controls report a strategy on all 16, and every arm's out-of-sample confidence interval includes zero. Supporting studies find that state-based stopping reaches the same outcome after 8 to 9 trials at about a twelfth of the trial budget while an adaptive budget adds nothing, that a controlled stress test reproduces the dilution effect and both defences, and that a hundred nominal trials collapse to 6 to 8 effectively independent return streams. We scope these results to the analysed system.

stat.AP↗

What survives honest evaluation? Leakage-safe, search-aware assessment of LLM-driven trading strategy discovery

Large language models (LLMs) are increasingly used to discover trading strategies, and much of the resulting literature shares a methodological weakness: many candidate strategies are generated, the best is reported, and neither look-ahead bias nor the intensity of the search behind the reported result is corrected for. We present a strategy-discovery system that makes both corrections structural rather than procedural. First, the agent can only act through registry-validated tools whose feature space excludes look-ahead by construction; we show that this guardrail is not redundant with statistical correction: a deliberately leaky oracle posting a Sharpe ratio of 35 survives Deflated Sharpe and probability-of-backtest-overfitting testing completely. Second, the system records every strategy evaluation its search performs and deflates all reported performance by that trial count, tracing how the best in-sample Sharpe ratio climbs with each trial while the deflation threshold, driven by the agent's own search, climbs faster. Across a 453-stock point-in-time US equity universe and a 39-ETF multi-asset universe with realistic transaction, impact, and borrow costs, honest evaluation certifies passive benchmarks (out-of-sample confidence intervals excluding zero), rejects every LLM-discovered strategy (across two frontier models, search budgets up to one hundred candidates, and five repeated runs), catching selection luck, predicted rank degradation, and out-of-sample collapse through complementary instruments, and evaluates a human trader's production rule system under identical instruments. The framework formalizes why pre-registered hypotheses earn lower evidential bars than brute search, and quantifies the sample sizes that credible certification of moderate edges actually requires.

q-fin.ST↗