arXiv subjects

Lior Horesh

Publications and source records attributed to Lior Horesh.

At least 19 recordsLinked to original sources

arXiv2026

A stability order barrier and SSP bound for Runge--Kutta methods with non-negative coefficients: AI-assisted proofs

We prove three results about Runge--Kutta methods with non-negative coefficients. First, every $s$-stage implicit Runge--Kutta method of order at least two has SSP coefficient at most $2s$; the bound is sharp and is attained by a known diagonally implicit method, so full implicitness gives no advantage over DIRK at second order. This extends the second-order DIRK result of Kocsis \& Németh to full coefficient matrices and settles a conjecture of Ketcheson, Macdonald & Gottlieb. These proofs were found with the assistance of AI tools, which uncovered an interesting relationship between the Butcher matrices for SSP methods and phase-type distributions, and led to the proof of the second result: a bound on the SSP coefficient of $s$-stage implicit Runge--Kutta method of order at least three. The third result in this work is a proof that a third-order Runge--Kutta method with entrywise non-negative $A$ and $b$ cannot be algebraically stable. This algebraically intensive proof was also AI-assisted. This paper describes the iterative process of AI-assisted results, and presents an approach that centers human understanding and creativity in the AI-assisted mathematical endeavor.

math.NA↗

arXiv2026

Leaner Transformers Can Easily Learn to Cluster

Transformers have in-context learning capabilities, where some known learning algorithms can be executed in the forward pass through the model. Recent work shows that transformers can exactly perform Lloyd's algorithm for $k$-means clustering with $n$ points in $d$ dimensions with an embedding size $d_{\textsf{emb}} = d+k$ (thus, requiring attention projection matrices of size $(d+k)^2$). In this work, we build upon this result in the following ways: First, we present an equally expressive but smaller transformer that executes Lloyd's algorithm with embedding size $d_{\textsf{emb}} = (d + \lceil \log_2 k \rceil)$. Next, we train these transformers to learn the clustering algorithms given a distribution of clustering tasks, and theoretically characterize and empirically validate the factors affecting the convergence and in-distribution generalization of learning algorithms based on stochastic gradients. Finally, we probe the general clustering abilities of these learned algorithms (in the form of transformers), and try to understand situations where they succeed and fail.

cs.LG↗

arXiv2026

When is a System Discoverable from Data? Discovery Requires Chaos

The deep learning revolution has enabled major AI advances in physical sciences, particularly in discovering dynamical systems from observational data. Yet the reliability of learned surrogates and symbolically discovered models is often undermined by non-uniqueness: models may fit observed trajectories perfectly, but lack genuine predictive power. We show that chaos, despite its association with unpredictability, is often what makes governing equations discoverable. We prove that the classical Lorenz system is analytically discoverable and verify experimentally that for many systems out-of-domain generalization emerges precisely at chaos onset. Conservative systems, by contrast, are inherently non-discoverable from data alone. The prevalence of chaotic systems in benchmarks may have inadvertently obscured this limitation. These findings help explain why AI excels at weather forecasting, yet struggles in many engineering applications: the systems most amenable to discovery (chaotic ones) are often least amenable to control.

math.DS↗

arXiv2026

Exact Symmetry as Algebra: Group-Structured Tensors that Enforce Physical Selection Rules

Symmetry determines which physical responses of molecules and crystals are allowed and underlies many models used throughout the physical sciences. Yet several widely used machine-learned models learn symmetry only approximately from data, raising the question of whether approximate symmetry is sufficient for scientific prediction. Using the Orb-v3 force field, we show that it is often not: the relaxed space group of a crystal depends on the orientation of the input for $47\%$ of a $150$-crystal subsample, and relaxation lowers the space group for half of all $399$ crystals studied. To characterize, quantify, and correct such failures, we introduce a tensor algebra, $\star_G$, defined by convolution over a finite symmetry group and applicable to tensors with noncommuting entries. We prove that the symmetry group determines the algebra's block-diagonal transform and use this structure to analyze model errors on symmetric targets. The resulting projections decompose the average squared error into orthogonal components arising from selection-rule violations, inconsistencies across symmetry-related inputs, and residual error. The removable components reveal how much error arises from broken symmetry, provide reference-free lower bounds on prediction error, and can be eliminated without sacrificing accuracy. For operators, projection followed by blockwise truncation yields the optimal symmetric low-rank approximation. Applying these projections to force predictions reduces both rates of space-group change to at most $2\%$. Exact selection rules, reference-free error bounds, and symmetry-preserving corrections are therefore available for any trained model without retraining.

cs.LG↗

arXiv2026

Transformer Circuits Can Realize Clustering Algorithms

Although transformers are most commonly optimized as statistical sequence models, it is unclear to what extent they can implement and learn exact algorithmic computations. Here, we specify a transformer implementation from first principles that executes a fundamental and widely used method for $k$-means clustering: Lloyd's algorithm. We theoretically prove and empirically demonstrate that this implementation of a transformer architecture, which we term the $k$-means transformer, exactly implements Lloyd's algorithm for $k$-means clustering using the standard circuit mechanisms of modern transformers: attention block, residual connections, and feed-forward block. In learning experiments, we find that training this base architecture on $k$-means clustering yields a generalizable clustering algorithm that surpasses Lloyd's algorithm in terms of clustering quality. Finally, we demonstrate that interpretable alterations (e.g., inclusion of layer normalizations) to this architecture yields diverse and novel variants of clustering algorithms, including soft $k$-means, spherical $k$-means, trimmed $k$-means. Overall, our results show that transformer circuit mechanisms can instantiate exact algorithmic routines for clustering, while simultaneously providing an effective learnable model.

cs.LG↗

arXiv2026

Counting Triangles of Graphs via Randomized Trace Estimation with Incomplete Matrix-Vector Products

Counting triangles in graphs is a fundamental operation in network analysis, underpinning metrics such as clustering coefficients and serving as a signal for community detection, link prediction, and anomaly detection. The standard approach computes the trace of the cube of the adjacency matrix, but explicitly forming $\mathbf{A}^3$ is infeasible for large graphs. Hutchinson randomized trace estimator offers an efficient alternative by approximating the trace through stochastic averaging of quadratic forms, requiring only matrix vector products with $\mathbf{A}$. However, in distributed and heterogeneous computing environments, observing all entries of these products can be costly due to communication overhead and straggler effects. To address this, we propose a new variant of Hutchinson estimator that operates under partial observation constraints, where both the number and identities of observed entries are random. We provide theoretical guarantees on unbiasedness, variance bounds, and sample complexity, and demonstrate through experiments on synthetic and real world graphs that our method achieves accurate triangle count estimates while reducing synchronization costs. This work highlights the adaptability of randomized algorithms to modern computational architectures and opens avenues for efficient motif counting in large scale network analytics.

math.NA↗

arXiv2026

Hybrid Digital-Analog Approximate Inverse Preconditioning for Krylov Methods

Analog in-memory computing enables highly parallel matrix-vector multiplications with reduced data movement, but the resulting operations are noisy, quantized, and affected by device- and circuit-level non-idealities. This paper studies approximate inverse preconditioning for Krylov subspace methods in a hybrid digital-analog setting. The digital host performs sparse products with the coefficient matrix and the precision-sensitive Krylov operations, while preconditioner applications are performed through analog crossbar matrix-vector multiplications. Since the realized preconditioner is inexact and application-dependent, the outer iteration is formulated as the flexible GMRES method. We show that analog execution changes the usual preconditioner design problem in the sense that a stronger digital preconditioner may be less effective after analog scaling, write noise, input/output perturbations, quantization, and clipping are taken into account. We compare various block Jacobi preconditioning schemes including exact block inverses, sparse approximate inverses, Monte Carlo approximate inverses (MCAI), damping, and nested block Jacobi schemes. Numerical experiments with realistic analog matrix-vector simulations show that analog-aware choices of block size, damping, MCAI construction accuracy, and nesting are important for robust convergence.

math.NA↗

arXiv2026

Analysis of Power Iteration Algorithm with Partially Observed Matrix-vector Products

We consider the problem of computing the dominant eigenvector of a symmetric matrix via the power iteration algorithm subject to constraints in the computation of matrix-vector pr ucts. In particular, we focus on scenarios where the entries of matrix-vector products with the input matrix are only partially observed. Such constraints frequently arise on cloud architectures implemented via the controller-worker model where the matrix-vector products are distributed across workers on remote servers. Instead of a prolonged delay incurred by waiting for the slowest workers to return their output to the controller, a phenomenon known as straggling, a set of pre-determined values can replace the values of the delayed workers and allow the power iteration to proceed to the next iteration. In this paper, we develop two algorithms whose expected approximation converges to the true dominant eigenvector. The first algorithm relies on a probabilistic switch between two different approaches to set the omitted entries: either set them to zero or to their previous recorded value. The second algorithm relies on averaging previously generated partial power iteration approximations obtained by ignoring a set of columns of the iteration matrix. several theoretical details are discussed while numerical experiments verify the effectiveness of the two proposed schemes and demonstrate their comparative performance advantage over current state-of-the-art.

math.NA↗

arXiv2026

A Differentiable Measure of Algebraic Complexity: Provably Exact Discovery of Group Structures

Discovering discrete algebraic rules from data is a fundamental challenge in machine learning. We formalize this problem through Cayley-table completion -- an algebraic counterpart to classical matrix completion -- where the degree of associativity violation replaces linear rank as the intrinsic measure of complexity. We provide a rigorous landscape analysis of HyperCube, an operator-valued tensor factorization, on the fully observed target table $δ$, proving that its global infimum $H_{\inf}(δ) := \inf_{Θ\in F_δ} H(Θ)$ implicitly defines an exact differentiable measure for this complexity. We show that HyperCube's native objective $H(Θ)$ decomposes into two components: geometric alignment (collinearity) and an inverse $\ell_2$ penalty. We establish that these continuous variational pressures induce core discrete properties: collinearity enforces associativity (Collinearity--Associativity Equivalence), and the inverse $\ell_2$ penalty reduces to an exact inverse rank penalty within the collinear manifold, driving the parameters toward full-rank unitarity. Consequently, we derive an absolute lower bound $H(Θ) \ge H_{\inf}(δ) \ge 3 \, |δ|$, where $|δ|$ is the target table size. We prove this absolute floor is attained if and only if the target is isotopic to a group, and characterize the global minimizer as the regular representation of the underlying group (up to unitary gauge), resolving the central open conjecture of Huh (2025). This work serves as an existence proof that certain discrete algebraic structures can be exactly characterized by differentiable measures, enabling gradient-based discovery without the need for combinatorial search. All theoretical results are mechanically verified in Lean 4 and confirmed via small-scale experiments.

cs.LG↗

arXiv2026

Interpretable epistemic uncertainty decomposition in sequential generative models via polynomial chaos surrogates

Sequential generative models conditioned on uncertain rewards are central to AI-driven scientific discovery, yet the epistemic uncertainty they inherit from imperfect reward estimates remains unquantified. We propagate this uncertainty through generative flow networks (GFlowNets) by fitting polynomial chaos expansions (PCEs) to small ensembles of trained models. The PCE coefficients yield analytical Sobol sensitivity indices, providing the first interpretable decomposition of which reward components drive which generative decisions, a capability unavailable from deep ensembles, Bayesian neural networks, or Monte Carlo dropout. Convergence guarantees are established theoretically and four of five are formally verified in the Lean 4 proof assistant. Across three real-world tasks the framework reveals actionable structure invisible to ensembles alone. On the Doyle-Dreher Buchwald-Hartwig dataset catalyst selection is robust ($D_{\mathrm{catalyst}}\approx 71$) while additive selection is fragile ($D_{\mathrm{additive}}\approx 179$, $2.5\times$ higher). In fragment-based molecular design the linker position is the most sensitive ($D_{\mathrm{linker}}\approx 28$) while decoration positions are the most robust ($D\approx 14$-$18$), reversing the conventional scaffold-robust / decoration-fragile assumption. On the Sachs protein signalling network, MAPK-cascade edges and PKA/PKC hub edges separate into distinct sensitivity regimes, providing a targeted map for perturbation experiments. Calibration coverage at the 95% level reaches 0.97-1.00 across the dominant steps, and the surrogate evaluates 10{,}000 policy samples in milliseconds - $10^{3}$-$10^{4}\times$ faster than exhaustive retraining.

cs.LG↗

arXiv2026

SynPAT: A System for Generating Synthetic Physical Theories with Data

Machine-assisted methods for discovering physical laws from background theory and data have recently emerged, promising to advance our understanding of the physical world. However, training and benchmarking these systems remains challenging: real physical theories are limited in number. To address this need, we introduce SynPAT, a system for generating synthetic physical theories with accompanying data. SynPAT produces: (i) a consistent set of axioms forming a synthetic theory, (ii) a symbolic consequence of these axioms representing the discovery target, and (iii) noisy data approximating this consequence. Crucially, to mirror historically incorrect theories (e.g., Newtonian mechanics before Special Relativity), SynPAT can also generate theories whose axioms do not strictly entail, and in fact conflict with, the observed consequence, requiring a correction to the assumed axioms to bridge the gap. We detail SynPAT's methodology and benchmark several open-source symbolic regression systems on our generated theories and data.

cs.SC↗

arXiv2025

Bridging the Gap Between Scientific Laws Derived by AI Systems and Canonical Knowledge via Abductive Inference with AI-Noether

Advances in AI have shown great potential in contributing to the acceleration of scientific discovery. Symbolic regression can fit interpretable models to data, but these models are not necessarily derivable from established theory. Recent systems (e.g., AI-Descartes, AI-Hilbert) enforce derivability from prior knowledge. However, when existing theories are incomplete or incorrect, these machine-generated hypotheses may fall outside the theoretical scope. Automatically finding corrections to axiom systems to close this gap remains a central challenge in scientific discovery. We propose a solution: an open-source algebraic geometry-based system that, given an incomplete axiom system expressible as polynomials and a hypothesis that the axioms cannot derive, generates a minimal set of candidate axioms that, when added to the theory, provably derive the (possibly noisy) hypothesis. We illustrate the efficacy of our approach by showing that it can reconstruct key axioms required to derive the carrier-resolved photo-Hall effect, Einstein's relativistic laws, and several other laws.

cs.AI↗

arXiv2025

The Need for Verification in AI-Driven Scientific Discovery

Artificial intelligence (AI) is transforming the practice of science. Machine learning and large language models (LLMs) can generate hypotheses at a scale and speed far exceeding traditional methods, offering the potential to accelerate discovery across diverse fields. However, the abundance of hypotheses introduces a critical challenge: without scalable and reliable mechanisms for verification, scientific progress risks being hindered rather than being advanced. In this article, we trace the historical development of scientific discovery, examine how AI is reshaping established practices for scientific discovery, and review the principal approaches, ranging from data-driven methods and knowledge-aware neural architectures to symbolic reasoning frameworks and LLM agents. While these systems can uncover patterns and propose candidate laws, their scientific value ultimately depends on rigorous and transparent verification, which we argue must be the cornerstone of AI-assisted discovery.

cs.AI↗

arXiv2025

Comparing quantum and classical Monte Carlo algorithms for estimating Betti numbers of clique complexes

Several quantum and classical Monte Carlo algorithms for Betti Number Estimation (BNE) on clique complexes have recently been proposed, though it is unclear how their performances compare. We review these algorithms, emphasising their common Monte Carlo structure within a new modular framework. We derive upper bounds for the number of samples needed to reach a given level of precision, and use them to compare these algorithms. By recombining the different modules, we create a new quantum algorithm with an exponentially-improved dependence in the sample complexity. We run classical simulations to verify convergence within the theoretical bounds and observe the predicted exponential separation, even though empirical convergence occurs substantially earlier than the conservative theoretical bounds.

quant-ph↗

arXiv2025

Fast Linear Solvers via AI-Tuned Markov Chain Monte Carlo-based Matrix Inversion

Large, sparse linear systems are pervasive in modern science and engineering, and Krylov subspace solvers are an established means of solving them. Yet convergence can be slow for ill-conditioned matrices, so practical deployments usually require preconditioners. Markov chain Monte Carlo (MCMC)-based matrix inversion can generate such preconditioners and accelerate Krylov iterations, but its effectiveness depends on parameters whose optima vary across matrices; manual or grid search is costly. We present an AI-driven framework recommending MCMC parameters for a given linear system. A graph neural surrogate predicts preconditioning speed from $A$ and MCMC parameters. A Bayesian acquisition function then chooses the parameter sets most likely to minimise iterations. On a previously unseen ill-conditioned system, the framework achieves better preconditioning with 50\% of the search budget of conventional methods, yielding about a 10\% reduction in iterations to convergence. These results suggest a route for incorporating MCMC-based preconditioners into large-scale systems.

cs.LG↗

arXiv2025

Language Models Coupled with Metacognition Can Outperform Reasoning Models

Large language models (LLMs) excel in speed and adaptability across various reasoning tasks, but they often struggle when strict logic or constraint enforcement is required. In contrast, Large Reasoning Models (LRMs) are specifically designed for complex, step-by-step reasoning, although they come with significant computational costs and slower inference times. To address these trade-offs, we employ and generalize the SOFAI (Slow and Fast AI) cognitive architecture into SOFAI-LM, which coordinates a fast LLM with a slower but more powerful LRM through metacognition. The metacognitive module actively monitors the LLM's performance and provides targeted, iterative feedback with relevant examples. This enables the LLM to progressively refine its solutions without requiring the need for additional model fine-tuning. Extensive experiments on graph coloring and code debugging problems demonstrate that our feedback-driven approach significantly enhances the problem-solving capabilities of the LLM. In many instances, it achieves performance levels that match or even exceed those of standalone LRMs while requiring considerably less time. Additionally, when the LLM and feedback mechanism alone are insufficient, we engage the LRM by providing appropriate information collected during the LLM's feedback loop, tailored to the specific characteristics of the problem domain and leads to improved overall performance. Evaluations on two contrasting domains: graph coloring, requiring globally consistent solutions, and code debugging, demanding localized fixes, demonstrate that SOFAI-LM enables LLMs to match or outperform standalone LRMs in accuracy while maintaining significantly lower inference time.

cs.AI↗

arXiv2025

Regenerative Ulam-von Neumann Algorithm: An Innovative Markov chain Monte Carlo Method for Matrix Inversion

This paper presents a regenerative variant of the classical Ulam-von Neumann Markov chain Monte Carlo algorithm for the approximation of the matrix inverse. The algorithm presented in this paper, termed regenerative Ulam-von Neumann algorithm, utilizes the regenerative structure of classical, non-truncated Neumann series defined by a non-singular matrix and produces an estimator of the matrix inverse via ratios of unbiased estimators of the regenerative quantities. The accuracy of the proposed algorithm depends on a single parameter that controls the total number of simulated Markov transitions, thus avoiding the challenge of balancing between the total number of Markov chain replications and their length as in the classical Ulam-von Neumann algorithm. To efficiently utilize Markov chain transition samples in the calculation of the regenerative variables, the proposed algorithm automatically quantifies the contribution of each Markov transition to all regenerative quantities by a carefully designed updating scheme that utilized three separate matrices containing the current weights, total weights, and regenerative cycle count, respectively. A probabilistic analysis of the performance of the algorithm, including the variance of the estimator, is provided. Finally, numerical experiments verify the effectiveness of the proposed scheme.

math.NA↗

arXiv2025

PCENet: High Dimensional Surrogate Modeling for Learning Uncertainty

Learning data representations under uncertainty is an important task that emerges in numerous scientific computing and data analysis applications. However, uncertainty quantification techniques are computationally intensive and become prohibitively expensive for high-dimensional data. In this study, we introduce a dimensionality reduction surrogate modeling (DRSM) approach for representation learning and uncertainty quantification that aims to deal with data of moderate to high dimensions. The approach involves a two-stage learning process: 1) employing a variational autoencoder to learn a low-dimensional representation of the input data distribution; and 2) harnessing polynomial chaos expansion (PCE) formulation to map the low dimensional distribution to the output target. The model enables us to (a) capture the system dynamics efficiently in the low-dimensional latent space, (b) learn under uncertainty, a representation of the data and a mapping between input and output distributions, (c) estimate this uncertainty in the high-dimensional data system, and (d) match high-order moments of the output distribution; without any prior statistical assumptions on the data. Numerical results are presented to illustrate the performance of the proposed method.

cs.LG↗