Benjamin Bradley | Searcharxiv

arXiv2026

Block-Based Double Decoders

Encoder-decoder models offer substantial inference-time savings over decoder-only models, but their pretraining objectives suffer from sparse supervision and dynamic sequence lengths, keeping them out of practice at scale. We propose block-based double decoders, a novel transformer architecture that utilizes doubly-causal block-based attention masks to train with full loss supervision and static sequence packing, combining decoder-only training efficiency with encoder-decoder inference efficiency. In scaling law experiments, block-based double decoders strongly outperform encoder-decoders and closely track decoder-only models across scales. At inference time, they cut KV-cache memory and per-token compute by at least 2/3 without sacrificing prefill caching or other existing inference optimizations available to decoder-only models.

cs.LG↗

arXiv2026

Autonomous Discovery of Particle Physics Theories from Experimental Data

The search for physics beyond the Standard Model is hindered by a combinatorial explosion of possible theories. We introduce \textsc{Albert}, a neuro-symbolic artificial intelligence framework to systematically navigate this vast theory space. By encoding particle physics as a formal language, \textsc{Albert} generates tokenized sequences representing symmetries, particles, and interactions under a rule-based grammar, eliminating the hallucinations common in large language models. The reinforcement learning environment enforces first-principle theoretical constraints, computes observables with radiative corrections, and evaluates statistical likelihood via $\chi^2$ analysis against experimental data. As a proof of concept, we train a 25-million-parameter transformer model using only legacy data from the Large Electron-Positron Collider, which contains no direct evidence of the top quark. Remarkably, \textsc{Albert} successfully rediscovered the Standard Model and autonomously inferred necessity and properties of the top quark, predicting its mass at $178.9\pm 5.0~\text{GeV}$, consistent with its modern measurement at the Large Hadron Collider. These results demonstrate the potential of AI-driven theory exploration as a rigorous, hallucination-free, and scalable paradigm for autonomous discovery of new physics.

hep-ph↗