arXiv subjects

Darren Li

Publications and source records attributed to Darren Li.

3 recordsLinked to original sources

arXiv2026

General learned delegation by clones

Frontier language models improve with additional test-time computation, but serial reasoning or uncoordinated parallel sampling can be compute-inefficient under fixed inference budgets. We propose SELFCEST, which equips a base model with the ability to spawn same-weight clones in separate parallel contexts by agentic reinforcement learning. Training is end-to-end under a global task reward with shared-parameter rollouts, yielding a learned controller that allocates both generation and context budget across branches. Across challenging math reasoning benchmarks and long-context multi-hop QA, SELFCEST improves the accuracy-cost Pareto frontier relative to monolithic baselines at matched inference budget, and exhibits out-of-distribution generalization in both domains.

cs.AI↗

arXiv2025

Continuous Autoregressive Language Models

The efficiency of large language models (LLMs) is fundamentally limited by their sequential, token-by-token generation process. We argue that overcoming this bottleneck requires a new design axis for LLM scaling: increasing the semantic bandwidth of each generative step. To this end, we introduce Continuous Autoregressive Language Models (CALM), a paradigm shift from discrete next-token prediction to continuous next-vector prediction. CALM uses a high-fidelity autoencoder to compress a chunk of K tokens into a single continuous vector, from which the original tokens can be reconstructed with over 99.9\% accuracy. This allows us to model language as a sequence of continuous vectors instead of discrete tokens, which reduces the number of generative steps by a factor of K. The paradigm shift necessitates a new modeling toolkit; therefore, we develop a comprehensive likelihood-free framework that enables robust training, evaluation, and controllable sampling in the continuous domain. Experiments show that CALM significantly improves the performance-compute trade-off, achieving the performance of strong discrete baselines at a significantly lower computational cost. More importantly, these findings establish next-vector prediction as a powerful and scalable pathway towards ultra-efficient language models. Code: https://github.com/shaochenze/calm. Project: https://shaochenze.github.io/blog/2025/CALM.

cs.CL↗

arXiv2023

An Efficient Modular Exponentiation Proof Scheme

We present an efficient proof scheme for any instance of left-to-right modular exponentiation, used in many computational tests for primality. Specifically, we show that for any $(a,n,r,m)$ the correctness of a computation $a^n\equiv r\pmod m$ can be proven and verified with an overhead negligible compared to the computational cost of the exponentiation. Our work generalizes the Gerbicz-Pietrzak proof scheme used when $n$ is a power of $2$, and has been successfully implemented at PrimeGrid, doubling the efficiency of distributed searches for primes.

cs.CR↗