Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2609.40258

Large Language Model-Guided Evolutionary Discovery of Native Neural Architectures for Spiking Sequence Modeling

Ruoyu Zhao·Jiaqi Wu·Chenyu Zhu·Zhichao Lu
View original source ↗Read source document ↗Export metadata ↓

Abstract

Spiking neural networks (SNNs) offer low-energy sequence modeling through sparse, event-driven computation. However, interactions among spike encoding, neuronal dynamics, and information propagation complicate architecture design. Existing SNN sequence models often adapt artificial neural network (ANN) architectures designed for real-valued activations, potentially underusing spike-based communication and temporal state updates, motivating automated discovery of native SNN architectures. Most evolutionary neural architecture search (ENAS) methods operate within predefined configuration spaces, limiting discovery to mechanisms expressible within those spaces. We introduce OpenArchEvo, which uses large language models (LLMs) to evolve executable architecture code in an open program space under spiking-projection constraints. In this space, code differences need not reflect architectural novelty, while direct performance evaluation requires costly training. We construct a three-view representation spanning code, design rationale, and a behavioral fingerprint to support novelty estimation and performance prediction. The search treats predicted performance and estimated novelty as two objectives, using surrogate predictions to select candidates for expensive training evaluations. With an estimated candidate-training cost of 132 V100 GPU-days, the search uncovers multiple native SNN architectures, exemplified by three designs featuring mechanisms such as spike-activity-dependent control of state updates and residual pathways. The discovered NeuroGate surpasses the ANN DeltaNet on WikiText-103, and the discovered architectures reduce estimated architecture-level arithmetic energy by up to 50.6x (LoopMem) relative to a common dense Transformer (ANN) baseline. All code and all discovered architectures will be made publicly available soon.

Explore related subjects

cs.NE

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Ruoyu Zhao, Jiaqi Wu, Chenyu Zhu, Zhichao Lu. 2026-09-30. Large Language Model-Guided Evolutionary Discovery of Native Neural Architectures for Spiking Sequence Modeling. https://arxiv.org/abs/2609.40258

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Fairness-Aware Performance Evaluation for Multi-Party Multi-Objective Optimization

In multiparty multiobjective optimization problems, solution sets are usually evaluated using classical performance metrics, aggregated across DMs. However, such mean-based evaluations may be unfair by favoring certain parties, as they assume identical geometric approximation quality to each party's PF carries comparable evaluative significance. Moreover, prevailing notions of MPMOP optimal solutions are restricted to strictly common Pareto optimal solutions, representing a narrow form of cooperation in multiparty decision making scenarios. These limitations obscure whether a solution set reflects balanced relative gains or meaningful consensus among heterogeneous DMs. To address these issues, this paper develops a fairness-aware performance evaluation framework grounded in a generalized notion of consensus solutions. From a cooperative game-theoretic perspective, we formalize four axioms that a fairness-aware evaluation function for MPMOPs should satisfy. By introducing a concession rate vector to quantify acceptable compromises by individual DMs, we generalize the classical definition of MPMOP optimal solutions and embed classical performance metrics into a Nash-product-based evaluation framework, which is theoretically shown to satisfy all axioms. To support empirical validation, we further construct benchmark problems that extend existing MPMOP suites by incorporating consensus-deficient negotiation structures. Experimental results demonstrate that the proposed evaluation framework is able to distinguish algorithmic performance in a manner consistent with consensus-aware fairness considerations. Specifically, algorithms converging toward strictly common solutions are assigned higher evaluation scores when such solutions exist, whereas in the absence of strictly common solutions, algorithms that effectively cover the commonly acceptable region are more favorably evaluated.

cs.NE↗
arXiv2026

Novelty Search with Cross-Task Collaborative Discovery

Novelty Search (NS) promotes exploration by rewarding behaviorally novel solutions rather than directly optimizing predefined objectives, making it particularly useful when objective guidance is sparse or deceptive. However, existing NS methods are typically designed for a single search task. When multiple related NS tasks are considered independently, their search processes may repeatedly explore similar regions or rediscover solutions that could be useful across tasks, leading to redundant use of the evaluation budget. To address this issue, we formulate a multitask novelty search setting and propose Multitask Novelty Search with Cross-Task Collaborative Discovery (MTNS-CoD). The central idea is to coordinate discovery across tasks so that different task-specific searches explore complementary regions while useful discoveries can still be reused across tasks. Specifically, MTNS-CoD introduces a multitask repulsion mechanism to discourage redundant exploration in similar genotype regions, together with an adaptive inter-task transfer mechanism that adjusts transfer probabilities according to the observed utility of cross-task exchanges during evolution. Their joint use promotes complementary exploration while selectively reusing beneficial discoveries. We further extend MTNS-CoD to novelty-augmented optimization, where behavioral novelty and objective information are jointly considered to support exploration under deceptive objective landscapes. Experiments on synthetic benchmarks, deceptive maze navigation, MuJoCo policy optimization, and generative novelty search show that MTNS-CoD can improve behavioral discovery and search coverage over the considered single-task and multitask baselines, with additional benefits observed on problems containing deceptive objectives.

cs.NE↗
arXiv2026

Multi-Depth Temporal Fusion for Feedforward, Locally Trained Spiking Neural Networks

We propose a new spiking neural network (SNN) design to process static images and event streams using time-to-first-spike (TTFS) latencies. Our key research question is which architectural choices best accommodate local and online learning in multi-layer convolutional SNNs. This question is addressed via an original framework combining residual-like connections with multi-depth feature aggregation and consensus. The full SNN pipeline features an early-vision front end, to convert raw visual data into sparse spike latencies, a four-layer convolutional backbone trained layerwise with unsupervised spike-timing-dependent plasticity (STDP), a deterministic Multi-Depth Temporal Fusion (MDTF) and a final classifier trained with reward-modulated spike-timing-dependent plasticity (R-STDP). Rather than replacing early features in deeper layers, the proposed MDTF preserves early temporal evidence, adding sparse residual events from intermediate layers, and incorporating deeper features only when they agree in time with earlier representations. The resulting architecture is experimentally validated across MNIST, Fashion-MNIST, CIFAR-10, and N-MNIST, delivering strong classification performance under a fully local learning regime. Selective multi-depth fusion significantly outperforms traditional STDP/R-STDP baselines on higher-variability visual tasks (achieving +18.2 pp on Fashion-MNIST and +29.2 pp on CIFAR-10). Furthermore, activity-budget analyses show that the network retains high accuracy even when removing a large fraction of late or weak spike events, confirming its high data efficiency and reduced event-processing requirements. The codebase is publicly available at github.com/aidinattar/multi-depth-temporal-fusion-snn.

cs.NE↗