arXiv subjects

Timothee Leleu

Publications and source records attributed to Timothee Leleu.

9 recordsLinked to original sources

arXiv2026

LLM-Guided Neural Architecture Search for Robust Co-Design of Physical Neural Networks

Deploying neural networks on unconventional hardware demands architectures that co-optimize task accuracy and platform-specific constraints such as energy cost, physical non-idealities, and numerical precision. Existing neural architecture search (NAS) methods are typically tailored to a single hardware family, limiting cross-platform comparison and generalization. We introduce Unconventional Hardware Neural Architecture Search (UH-NAS), a hardware-agnostic, LLM-guided NAS framework that integrates language models as evolutionary operators to co-optimize accuracy and inference energy. By exposing hardware as a swappable backend with per-platform energy models, physical constraints, and non-ideality simulators, UH-NAS enables fair system-level comparisons across various backends without modifying the search algorithm. Tested on optical MZI hardware, UH-NAS discovers more diverse, robust architectures than conventional baselines while outperforming existing LLM-to-NAS approaches. Additional ablations on architecture robustness under non-idealities and the role of system prompts highlight the importance of architecture-hardware co-design for emerging computing platforms.

cs.LG↗

arXiv2026

Improving Generalization by Permutation Routing Across Model Copies

We introduce a use of the $M$-cover (or $M$-layer) transform for machine learning. The method replicates a model $M$ times, but instead of coupling the copies through parameter averaging or an explicit attractive force, as in replicated SGD or Elastic SGD, it rewires the contexts in which local learning messages are computed. Each local loss is evaluated on a routed model whose parameters are drawn from different copies according to permutations sampled from a structured mixing kernel $Q$. Training then uses the original local update rule, while the resulting learning messages are redistributed across the copies through these routed computational paths. Thus $Q$ defines a topology for message transport and controls the long-loop structure of the lifted factor graph. We formulate this construction for perceptrons, committee machines, and multilayer perceptrons, showing that the same principle applies from discrete models to differentiable neural networks. The resulting framework provides a mechanism for improving generalization through structured message sharing rather than replica collapse or parameter-space coupling.

cs.LG↗

arXiv2026

Contrastive Concept-Tree Search for LLM-Assisted Algorithm Discovery

Large language Model (LLM)-assisted algorithm discovery is an iterative, black-box optimization process over programs to approximatively solve a target task, where an LLM proposes candidate programs and an external evaluator provides task feedback. Despite intense recent research on the topic and promising results, how can the LLM internal representation of the space of possible programs be maximally exploited to improve performance is an open question. Here, we introduce Contrastive Concept-Tree Search (CCTS), which extracts a hierarchical concept representation from the generated programs and learns a contrastive concept model that guides parent selection. By reweighting parents using a likelihood-ratio score between high- and low-performing solutions, CCTS biases search toward useful concept combinations and away from misleading ones, providing guidance through an explicit concept hierarchy rather than the algorithm lineage constructed by the LLM. We show that CCTS improves search efficiency over fitness-based baselines and produces interpretable, task-specific concept trees across a benchmark of open Erd\H{o}s-type combinatorics problems. Our analysis indicates that the gains are driven largely by learning which concepts to avoid. We further validate these findings in a controlled synthetic algorithm-discovery environment, which reproduces qualitatively the search dynamics observed with the LLMs.

cs.LG↗

arXiv2026

Reshaping Global Loop Structure to Accelerate Local Optimization by Smoothing Rugged Landscapes

Probabilistic graphical models with frustration exhibit rugged energy landscapes that trap iterative optimization dynamics. These landscapes are shaped not only by local interactions, but crucially also by the global loop structure of the graph. The famous Bethe approximation treats the graph as a tree, effectively ignoring global structure, thereby limiting its effectiveness for optimization. Loop expansions capture such global structure in principle, but are often impractical due to combinatorial explosion. The $M$-layer construction provides an alternative: make $M$ copies of the graph and reconnect edges between them uniformly at random. This provides a controlled sequence of approximations from the original graph at $M=1$, to the Bethe approximation as $M \rightarrow \infty$. Here we generalize this construction by replacing uniform random rewiring with a structured mixing kernel $Q$ that sets the probability that any two layers are interconnected. As a result, the global loop structure can be shaped without modifying local interactions. We show that, after this copy-and-reconnect transformation, there exists a regime in which layer-to-layer fluctuations decay, increasing the probability of reaching the global minimum of the energy function of the original graph. This yields a highly general and practical tool for optimization. Using this approach, the computational cost required to reach these optimal solutions is reduced across sparse and dense Ising benchmarks, including spin glasses and planted instances. When combined with replica-exchange Monte Carlo, the same construction increases the polynomial-time algorithmic threshold for the maximum independent set problem. A cavity analysis shows that structured inter-layer coupling significantly smooths rugged energy landscapes by collapsing configurational complexity and suppressing many suboptimal metastable states.

cond-mat.dis-nn↗

arXiv2026

Neural Ising Machines via Unrolling and Zeroth-Order Training

We propose a data-driven heuristic for NP-hard Ising and Max-Cut optimization that learns the update rule of an iterative dynamical system. The method learns a shared, node-wise update rule that maps local interaction fields to spin updates, parameterized by a compact multilayer perceptron with a small number of parameters. Training is performed using a zeroth-order optimizer, since backpropagation through long, recurrent Ising-machine dynamics leads to unstable and poorly informative gradients. We call this approach a neural network parameterized Ising machine (NPIM). Despite its low parameter count, the learned dynamics recover effective algorithmic structure, including momentum-like behavior and time-varying schedules, enabling efficient search in highly non-convex energy landscapes. Across standard Ising and neural combinatorial optimization benchmarks, NPIM achieves competitive solution quality and time-to-solution relative to recent learning-based methods and strong classical Ising-machine heuristics.

cs.LG↗

arXiv2024

Dynamic Anisotropic Smoothing for Noisy Derivative-Free Optimization

We propose a novel algorithm that extends the methods of ball smoothing and Gaussian smoothing for noisy derivative-free optimization by accounting for the heterogeneous curvature of the objective function. The algorithm dynamically adapts the shape of the smoothing kernel to approximate the Hessian of the objective function around a local optimum. This approach significantly reduces the error in estimating the gradient from noisy evaluations through sampling. We demonstrate the efficacy of our method through numerical experiments on artificial problems. Additionally, we show improved performance when tuning NP-hard combinatorial optimization solvers compared to existing state-of-the-art heuristic derivative-free and Bayesian optimization methods.

cs.LG↗

arXiv2022

Experimental observation of chimera states in spiking neural networks based on degenerate optical parametric oscillators

We experimentally demonstrate that networks of identical photonic spiking neurons based on coupled degenerate parametric oscillators can show various chimera states, in which, depending on their local synchronization and desynchronization, different kinds of spiking dynamics can develop in a self-organized manner. Even when only a static interaction is implemented, through synchronized inputs from connected neurons, the spiking mode of photonic neurons can be spontaneously and adaptively changed between the Class-I and Class-II modes classified by A. L. Hodgkin. This spontaneous spiking-mode shift induces a significant change in the spiking frequency despite the all neurons having the same natural spiking frequency, which encourages the generation of chimera states. Controllability and self-organized flexibility of the spiking modes in the present system allow us to create an experimental platform to explore the nature of chimera states in neuromorphic spiking neural networks.

physics.optics↗

arXiv2020

Scaling advantage of nonrelaxational dynamics for high-performance combinatorial optimization

The development of physical simulators, called Ising machines, that sample from low energy states of the Ising Hamiltonian has the potential to drastically transform our ability to understand and control complex systems. However, most of the physical implementations of such machines have been based on a similar concept that is closely related to relaxational dynamics such as in simulated, mean-field, chaotic, and quantum annealing. We show that nonrelaxational dynamics that is associated with broken detailed balance and positive entropy production rate can accelerate the sampling of low energy states compared to that of conventional methods. By implementing such dynamics on field programmable gate array, we show that the nonrelaxational dynamics that we propose, called chaotic amplitude control, exhibits a scaling with problem size of the time to finding optimal solutions and its variance that is significantly smaller than that of relaxational schemes recently implemented on Ising machines.

physics.comp-ph↗

arXiv2018

Destabilization of local minima in analog spin systems by correction of amplitude heterogeneity

The relaxation of binary spins to analog values has been the subject of much debate in the field of statistical physics, neural networks, and more recently quantum computing, notably because the benefits of using an analog state for finding lower energy spin configurations are usually offset by the negative impact of the improper mapping of the energy function that results from the relaxation. We show that it is possible to destabilize trapping sets of analog states that correspond to local minima of the binary spin Hamiltonian by extending the phase space to include error signals that correct amplitude inhomogeneity of the analog spin states and controlling the divergence of their velocity. Performance of the proposed analog spin system in finding lower energy states is competitive against state-of-the-art heuristics.

nlin.CD↗