Searcharxiv⌕ Search

arXiv · 2611.15250

Multi-head Quantum Neural Networks: Replacing Transformer Blocks with NISQ Modules inside a 30-Billion-Parameter LLM

Abstract

We introduce the multi-head quantum neural network (QNN), a quantum module compatible with noisy intermediate-scale quantum (NISQ) devices that can replace transformer blocks inside pre-trained large language models (LLMs) with little loss in accuracy and with far fewer parameters than the blocks it replaces. Encoding a model's $D$ hidden neurons into a qubit register is the obstacle: amplitude encoding scales poorly in circuit depth and angle encoding in qubit count, so neither is practical at production-scale width. We instead partition the hidden space into $k$ disjoint heads, angle-encode each into an $n_q$-qubit register, process the heads in parallel, and couple them through constrained classical linear layers, the mixers. Performing surgery on Qwen3-30B-A3B-Instruct-2507, we show that a single multi-head QNN with $145\,281$ trainable parameters ($D=2048$, $k=32$, $n_q=8$, $L=4$) can replace a contiguous span of eight transformer blocks, displacing $16.33\%$ of the model's parameters. Ablations show that, for a single replaced block, the quantum heads (circuits followed by a learned readout), and not the classical mixers alone, recover the displaced function, whereas for fused spans the mixers alone come within $1$-$3\%$ of the full module in perplexity. Whether this benefit is specific to trained, entangling quantum circuits is not established: controls that switch the entangler off (two seeds), or freeze the circuit at its initial or at uniformly random angles (one seed), beat the full module on held-out perplexity, and a parameter-matched classical multilayer perceptron remains a few percent ahead of the full module. We train classically with exact statevector simulation, then execute the quantum heads of our fusion checkpoints on a $156$-qubit IBM Heron processor, showing that NISQ-scale quantum modules can be installed in a production-scale LLM and run on hardware.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Maforikan Amoussou, Borja Aizpurua, Sukhbinder Singh, Román Orús, Hans-Arno Jacobsen. 2026-11-11. Multi-head Quantum Neural Networks: Replacing Transformer Blocks with NISQ Modules inside a 30-Billion-Parameter LLM. https://searcharxiv.com/abs/2611.15250

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

A Diamond Quantum Oscilloscope

Quantum-enhanced metrology has achieved remarkable precision in parameter estimation, but extending these advantages to continuous time-dependent fields (waveform sensing) remains a formidable challenge due to the infinite-dimensional nature of the signal. Here, we report the first experimental realization of a room-temperature solid-state quantum oscilloscope based on a sequential quantum difference amplification strategy. Within a zero-order-hold reconstruction framework, our method completely bypasses the need for massive entanglement. Crucially, we demonstrate that the overall waveform reconstruction error unequivocally follows the theoretically predicted Heisenberg scaling of \(O(N^{-1/2})\), decisively surpassing the classical \(O(N^{-1/3})\) limit. This transformative precision requires approximately 30-fold fewer quantum resources than conventional methods to achieve the same waveform error. Furthermore, we demonstrate the reconstruction of a multicomponent waveform and distinctly resolve weak high-frequency components that are entirely drowned in the shot noise of classical sampling. These results transition quantum metrology from parameter estimation into a practical instrument, establishing a genuine quantum advantage in continuous waveform reconstruction and paving the way for critical applications such as nanoscale nuclear magnetic resonance imaging, where detecting extremely weak signals is crucial for resolving molecular structures.

quant-ph↗

Entanglement preservation and Clauser-Horne nonlocality in electromagnetically induced transparency quantum memories

Understanding how decoherence-induced loss affects entanglement and nonlocal correlations in quantum memories is important for quantum information science. Here we develop a quantitative description of photonic quantum states retrieved from electromagnetically induced transparency (EIT) memories in a ground-state-decoherence-limited regime. Using the dark-state-polariton formalism, we derive the attenuation factor of the retrieved probe mode from the EIT dynamics. This attenuation depends on the coupling field and ground-state coherence decay and is not introduced as an independent phenomenological loss parameter. Within a specified quantum-limited pure-loss model, we use this attenuation to obtain the reduced photonic density operator. We show that, in the symmetric case, the complete retrieved state of an initially path-entangled single photon remains entangled for any nonzero storage efficiency, despite acquiring a vacuum component. In contrast, violation of the unconditional Clauser-Horne inequality under displaced on/off detection without post-selection requires an intrinsic efficiency above 89.7%. We further derive the retrieved N-qubit photonic density operator for multiple independently operating EIT memories. These results establish a quantitative connection between EIT memory dynamics, retrieved quantum states, and operational nonlocality benchmarks under explicit channel assumptions.

quant-ph↗

Geometric encoding of turbulence for end-to-end quantum simulation

Simulating multiscale turbulence in engineering remains challenging. Any quantum advantage for such simulations requires efficient initial-state preparation. Yet exact loading of an arbitrary classical snapshot requires exponential quantum resources. We introduce "turbuloscope", a physics-informed geometric method that generates statistically representative turbulent initial states in a gate-based register from a parameterized Pauli-spinor multiscale distribution. Scale invariance renders conditional rotations approximately affine in Gray-code space, enabling linear-depth, ancilla-free preparation. The generalized Madelung transform and Hopf fibration map the encoded Pauli spinor to fluid observables and vortex tubes. The qubit count scales logarithmically with the Reynolds number, yielding an exponential reduction in preparation overhead relative to full-grid classical spectral synthesis. We classically emulate a 30-qubit preparation circuit and generate a turbulent field over one billion grid points at a scale-inferred Reynolds number of 35,000, exhibiting Kolmogorov's five-thirds energy spectrum, tangled vortex tubes, and intermittency. This method provides scalable initial states for quantum simulation of turbulence and multiscale systems.

quant-ph↗