arXiv subjects

Sukhbinder Singh

Publications and source records attributed to Sukhbinder Singh.

At least 19 recordsLinked to original sources

arXiv2026

Tensorization is a powerful but underexplored tool for compression and interpretability of neural networks

Tensorizing a neural network involves reshaping some or all of its dense weight matrices into higher-order tensors and approximating them using low-rank tensor network decompositions. This technique has shown promise as a model compression strategy for large-scale neural networks. However, despite encouraging empirical results, tensorized neural networks (TNNs) remain underutilized in mainstream deep learning. In this position paper, we offer a perspective on both the potential and current limitations of TNNs. We argue that TNNs represent a powerful yet underexplored framework for deep learning--one that deserves greater attention from both engineering and theoretical communities. Beyond compression, we highlight the value of TNNs as a flexible class of architectures with distinctive scaling properties and increased interpretability. A central feature of TNNs is the presence of bond indices, which introduce new latent spaces not found in conventional networks. These internal representations may provide deeper insight into the evolution of features across layers, potentially advancing the goals of mechanistic interpretability. We conclude by outlining several key research directions aimed at overcoming the practical barriers to scaling and adopting TNNs in modern deep learning workflows.

cs.LG↗

arXiv2026

Quantum Large Language Models via Tensor Network Disentanglers

We introduce a framework for seamlessly integrating quantum computing into pretrained large language models (LLMs). The key idea is to construct a hybrid quantum-classical representation that exactly reproduces the original model, providing a principled starting point from which quantum resources can only improve performance. Our approach replaces the weight matrices in self-attention and multilayer perceptron layers with two variational quantum circuits coupled to a matrix product operator (MPO). Tensor network disentanglers transfer much of each layer's information into the quantum circuits, enabling the remaining tensor network to be compressed to a bond-dimension-one MPO with over three orders of magnitude fewer classical parameters (in our experiments, from 110,592 to approximately 36 for the replaced layer) and less than a 0.3\% increase in perplexity. Training an added unitary adapter on top of this representation then surpasses the original model, reducing perplexity by up to 1.6\%. Finally, we validate the hybrid architecture on a real quantum processor, demonstrating a practical route towards quantum-enhanced language models.

quant-ph↗

arXiv2026

Pushing the Classical Frontier of 1D Fermi-Hubbard Quench Dynamics Beyond Current Quantum Simulations

Establishing quantum advantage requires comparison against the best achievable classical simulation. The Q-CTRL team recently simulated quench dynamics of the one-dimensional Fermi-Hubbard model on an IBM processor, completing a $L=60$ evolution to time $t=6$ in under three minutes and claiming a $3000\times$ speedup over classical Time-Dependent Variational Principle (TDVP) simulation at bond dimension $χ=4096$. Their classical benchmark required over 160 hours on a CPU cluster, failed to converge in the high-entanglement regime $t\in[5.2,6]$, and left the most challenging window of the experiment unverified. Here, we push the boundaries of classical simulation by exploiting the full $\mathrm{U}(1)\times\mathrm{SU}(2)$ symmetry of the Fermi-Hubbard Hamiltonian combined with GPU-accelerated tensor contractions. Reaching bond dimensions up to $χ\approx62{,}000$ on four NVIDIA H200 GPUs -- among the largest ever achieved in TDVP simulations and fifteen times larger than Q-CTRL's classical baseline -- we achieve fully converged results across the entire simulation window, including rigorous certification of the previously unresolved high-entanglement regime $t\in[5.2,6]$. We further advance the classical frontier to $t=7$, which lies beyond the quantum hardware experiment and any previously verified classical evolution of the full wavefunction. At the bond dimension comparable to Q-CTRL's best classical run, our GPU implementation completes in $\sim\!100$ minutes, directly reducing the claimed $3000\times$ quantum advantage to $\sim\!36\times$. These results substantially narrow the quantum-classical performance gap and establish a new standard for tensor-network benchmarking of large-scale quantum simulations.

quant-ph↗

arXiv2026

Fast Tensorization of Neural Networks via Slice-wise Feature Distillation

We propose a scalable tensorization framework for neural network compression based on slice-wise feature distillation. Unlike conventional tensor decomposition methods that rely on costly global finetuning, our approach decomposes the network into slices consisting of either individual layers or blocks (e.g., convolutional layers or MLPs), or small groups of consecutive layers, and tensorizes each slice independently to reproduce the intermediate representations of the original pretrained model. This modular strategy improves accuracy recovery, reduces data requirements, and enables efficient parallel optimization. Experiments on ResNet-34 show significant gains over conventional global tensorization, achieving near-lossless compression at moderate compression rates with faster optimization. Results on GPT-2 XL further demonstrate the scalability of the method and its applicability to large-scale models, particularly in distributed settings.

cs.LG↗

arXiv2026

Quantum-enhanced Large Language Models on Quantum Hardware via Cayley Unitary Adapters

Large language models (LLMs) have transformed artificial intelligence, yet classical architectures impose a fundamental constraint: every trainable parameter demands classical memory that scales unfavourably with model size. Quantum computing offers a qualitatively different pathway, but practical demonstrations on real hardware have remained elusive for models of practical relevance. Here we show that Cayley-parameterised unitary adapters -- quantum circuit blocks inserted into the frozen projection layers of pre-trained LLMs and executed on a 156-qubit IBM Quantum System Two superconducting processor -- improve the perplexity of Llama 3.1 8B, an 8-billion-parameter model in widespread use, by 1.4% with only 6,000 additional parameters and end-to-end inference validated on real Quantum Processing Unit (QPU). A systematic study on SmolLM2 (135M parameters), chosen for its tractability, reveals monotonically improving perplexity with unitary block dimension, 83% recovery of compression-induced degradation, and correct answers to questions that both classical baselines fail -- with a sharp noise-expressivity phase transition identifying the concrete path to quantum utility at larger qubit scales.

quant-ph↗

arXiv2026

Classical Neural Networks on Quantum Devices via Tensor Network Disentanglers: A Case Study in Image Classification

We address the problem of implementing bottleneck layers from classical pre-trained neural networks on a quantum computer, with the goal of exploring intrinsically quantum ansatz for representing large linear layers within hybrid classical-quantum models. Our approach begins with a compression step in which the target linear layer is represented as an effective matrix product operator (MPO) without degrading model performance. The MPO is then further disentangled into a more compact form. This enables a hybrid classical-quantum execution scheme, where the disentangling circuits are deployed on a quantum computer while the remainder of the network -- including the disentangled MPO -- runs on classical hardware. We introduce two complementary algorithms for MPO disentangling: (i) an explicitly disentangling variational method leveraging standard tensor-network optimization techniques, and (ii) an implicitly disentangling gradient-descent-based approach. We validate these methods through a proof-of-concept translation of simple classical neural networks for MNIST and CIFAR-10 image classification into a hybrid classical-quantum form.

quant-ph↗

arXiv2026

Quantum Advantage: a Tensor Network Perspective

We review the recent quantum advantage experiments by IBM, D-Wave, and Google, focusing on cases where efficient classical simulations of the experiment were demonstrated or attempted using tensor network methods. We assess the strengths and limitations of these tensor network-based approaches and examine how the interplay between classical simulation and quantum hardware has advanced both fields. Our goal is to clarify what these results imply for the next generation of quantum advantage experiments. We identify regimes and system features that remain challenging for current tensor network approaches, and we outline directions where improved classical methods could further raise the standard for claiming quantum advantage. By analyzing this evolving competition, we aim to provide a clear view of where genuine, scalable quantum advantage is most likely to emerge.

quant-ph↗

arXiv2026

Only relative ranks matter in weight-clustered large language models

Large language models (LLMs) contain billions of parameters, yet many exact values are not essential. We show that what matters most is the relative rank of weights-whether one connection is stronger or weaker than another-rather than precise magnitudes. To reduce the number of unique weight values, we apply weight clustering to pretrained models, replacing every weight matrix with K shared values from K-means. For Llama 3.1-8B-Instruct and SmolLM2-135M, reducing each matrix to only 16-64 distinct values preserves strong accuracy without retraining, providing a simple, training-free method to compress LLMs on disk. Optionally fine-tuning only the cluster means (centroids) recovers 30-40 percent of the remaining accuracy gap at minimal cost. We then systematically randomize cluster means while keeping assignments fixed. Scrambling the relative ranks of the clusters degrades quality sharply-perplexity can increase by orders of magnitude-even when global statistics such as mean and variance are preserved. In contrast, rank-preserving randomizations cause almost no loss at mid and late layers. On the other hand, when many layers are perturbed simultaneously, progressive layer-by-layer replacement reveals that scale drift-not rank distortion-is the dominant collapse mechanism; however, an affine correction w' = aw + b with a > 0 (which preserves both rank order and overall weight distribution) can substantially delay this drift. This rank-based perspective offers a new lens on model compression and robustness.

cs.LG↗

arXiv2025

Globally optimized SVD compression of LLMs via Fermi-function-based rank selection and gauge fixing

Large Language Models (LLMs) are very demanding in terms of their computational resources. Low-rank decompositions of LLM weights, e.g. via Singular Value Decomposition (SVD), is a promising approach for LLM compression, but presents several practical hurdles, e.g. selecting appropriate layer-wise ranks and getting rid of its parameter redundancy. In this work, we present two physics-inspired improvements to SVD LLM compression: (1) \textbf{FermiGrad}, a gradient-descent algorithm that determines globally optimal layer-wise ranks by relaxing the discrete singular-value truncation into a continuous optimization using the Fermi function; (2) \textbf{PivGa}, an additional \textit{lossless} compression of the low-rank factors that exploits the intrinsic gauge freedom in their parametrization.

cs.LG↗

arXiv2024

Capturing long-range memory structures with tree-geometry process tensors

We introduce a class of quantum non-Markovian processes -- dubbed process trees -- that exhibit polynomially decaying temporal correlations and memory distributed across time scales. This class of processes is described by a tensor network with tree-like geometry whose component tensors are (1) causality-preserving maps (superprocesses) and (2) locality-preserving temporal change of scale transformations. We show that the long-range correlations in this class of processes tends to originate almost entirely from memory effects, and can accommodate genuinely quantum power-law correlations in time. Importantly, this class allows efficient computation of multi-time correlation functions. To showcase the potential utility of this model-agnostic class for numerical simulation of physical models, we show how it can efficiently approximate the strong memory dynamics of the paradigmatic spin-boson model, in terms of arbitrary multitime features. In contrast to an equivalently costly matrix product operator (MPO) representation, the ansatz produces a fiducial characterization of the relevant physics. Finally, leveraging 2D tensor network renormalization group methods, we detail an algorithm for deriving a process tree from an underlying Hamiltonian, via the Feynmann-Vernon influence functional. Our work lays the foundation for the development of more efficient numerical techniques in the field of strongly interacting open quantum systems, as well as the theoretical development of a temporal renormalization group scheme.

quant-ph↗

arXiv2024

Categorical Quantum Volume Operator

We present a generalization of the quantum volume operator quantifying the volume in curved three-dimensional discrete geometries. In its standard form, the quantum volume operator is constructed from tetrahedra whose faces are endowed with irreducible representations of $\mathrm{SU}(2)$. Here, we show two equivalent constructions that allow general objects in fusion categories as degrees of freedom. First, we compute the volume operator for ribbon fusion categories. This includes the important class of modular tensor categories (such as quantum doubles), which are the building blocks of anyon models. Second, we further generalize the volume operator to spherical fusion categories by relaxing the categorical analog of the closure constraint (known as tetrahedral symmetry). In both cases, we obtain a volume operator that is Hermitian, provided that the input category is unitary. As an illustrative example, we consider the case of $\mathrm{SU}(2)_k$ and show that the standard $\mathrm{SU}(2)$ volume operator is recovered in the limit $k\rightarrow\infty$.

gr-qc↗

arXiv2024

Tensor network compressibility of convolutional models

Convolutional neural networks (CNNs) are one of the most widely used neural network architectures, showcasing state-of-the-art performance in computer vision tasks. Although larger CNNs generally exhibit higher accuracy, their size can be effectively reduced by ``tensorization'' while maintaining accuracy, namely, replacing the convolution kernels with compact decompositions such as Tucker, Canonical Polyadic decompositions, or quantum-inspired decompositions such as matrix product states, and directly training the factors in the decompositions to bias the learning towards low-rank decompositions. But why doesn't tensorization seem to impact the accuracy adversely? We explore this by assessing how \textit{truncating} the convolution kernels of \textit{dense} (untensorized) CNNs impact their accuracy. Specifically, we truncated the kernels of (i) a vanilla four-layer CNN and (ii) ResNet-50 pre-trained for image classification on CIFAR-10 and CIFAR-100 datasets. We found that kernels (especially those inside deeper layers) could often be truncated along several cuts resulting in significant loss in kernel norm but not in classification accuracy. This suggests that such ``correlation compression'' (underlying tensorization) is an intrinsic feature of how information is encoded in dense CNNs. We also found that aggressively truncated models could often recover the pre-truncation accuracy after only a few epochs of re-training, suggesting that compressing the internal correlations of convolution layers does not often transport the model to a worse minimum. Our results can be applied to tensorize and compress CNN models more effectively.

cs.CV↗

arXiv2024

Projected Entangled Pair States with flexible geometry

Projected Entangled Pair States (PEPS) are a class of quantum many-body states that generalize Matrix Product States for one-dimensional systems to higher dimensions. In recent years, PEPS have advanced understanding of strongly correlated systems, especially in two dimensions, e.g., quantum spin liquids. Typically described by tensor networks on regular lattices (e.g., square, cubic), PEPS have also been adapted for irregular graphs, however, the computational cost becomes prohibitive for dense graphs with large vertex degrees. In this paper, we present a PEPS algorithm to simulate low-energy states and dynamics defined on arbitrary, fluctuating, and densely connected graphs. We introduce a cut-off, $κ\in \mathbb{N}$, to constrain the vertex degree of the PEPS to a set but tunable value, which is enforced in the optimization by applying a simple edge-deletion rule, allowing the geometry of the PEPS to change and adapt dynamically to the system's correlation structure. We benchmark our flexible PEPS algorithm with simulations of classical spin glasses and quantum annealing on densely connected graphs with hundreds of spins, and also study the impact of tuning $κ$ when simulating a uniform quantum spin model on a regular (square) lattice. Our work opens the way to apply tensor network algorithms to arbitrary, even fluctuating, background geometries.

cond-mat.str-el↗

arXiv2024

CompactifAI: Extreme Compression of Large Language Models using Quantum-Inspired Tensor Networks

Large Language Models (LLMs) such as ChatGPT and LlaMA are advancing rapidly in generative Artificial Intelligence (AI), but their immense size poses significant challenges, such as huge training and inference costs, substantial energy demands, and limitations for on-site deployment. Traditional compression methods such as pruning, distillation, and low-rank approximation focus on reducing the effective number of neurons in the network, while quantization focuses on reducing the numerical precision of individual weights to reduce the model size while keeping the number of neurons fixed. While these compression methods have been relatively successful in practice, there is no compelling reason to believe that truncating the number of neurons is an optimal strategy. In this context, this paper introduces CompactifAI, an innovative LLM compression approach using quantum-inspired Tensor Networks that focuses on the model's correlation space instead, allowing for a more controlled, refined and interpretable model compression. Our method is versatile and can be implemented with - or on top of - other compression techniques. As a benchmark, we demonstrate that a combination of CompactifAI with quantization allows to reduce a 93% the memory size of LlaMA 7B, reducing also 70% the number of parameters, accelerating 50% the training and 25% the inference times of the model, and just with a small accuracy drop of 2% - 3%, going much beyond of what is achievable today by other compression techniques. Our methods also allow to perform a refined layer sensitivity profiling, showing that deeper layers tend to be more suitable for tensor network compression, which is compatible with recent observations on the ineffectiveness of those layers for LLM performance. Our results imply that standard LLMs are, in fact, heavily overparametrized, and do not need to be large at all.

cs.CL↗

arXiv2024

Boosting Defect Detection in Manufacturing using Tensor Convolutional Neural Networks

Defect detection is one of the most important yet challenging tasks in the quality control stage in the manufacturing sector. In this work, we introduce a Tensor Convolutional Neural Network (T-CNN) and examine its performance on a real defect detection application in one of the components of the ultrasonic sensors produced at Robert Bosch's manufacturing plants. Our quantum-inspired T-CNN operates on a reduced model parameter space to substantially improve the training speed and performance of an equivalent CNN model without sacrificing accuracy. More specifically, we demonstrate how T-CNNs are able to reach the same performance as classical CNNs as measured by quality metrics, with up to fifteen times fewer parameters and 4% to 19% faster training times. Our results demonstrate that the T-CNN greatly outperforms the results of traditional human visual inspection, providing value in a current real application in manufacturing.

cs.CV↗

arXiv2024

Efficient tensor network simulation of IBM's largest quantum processors

We show how quantum-inspired 2d tensor networks can be used to efficiently and accurately simulate the largest quantum processors from IBM, namely Eagle (127 qubits), Osprey (433 qubits) and Condor (1121 qubits). We simulate the dynamics of a complex quantum many-body system -- specifically, the kicked Ising experiment considered recently by IBM in Nature 618, p. 500-505 (2023) -- using graph-based Projected Entangled Pair States (gPEPS), which was proposed by some of us in PRB 99, 195105 (2019). Our results show that simple tensor updates are already sufficient to achieve very large unprecedented accuracy with remarkably low computational resources for this model. Apart from simulating the original experiment for 127 qubits, we also extend our results to 433 and 1121 qubits, and for evolution times around 8 times longer, thus setting a benchmark for the newest IBM quantum machines. We also report accurate simulations for infinitely-many qubits. Our results show that gPEPS are a natural tool to efficiently simulate quantum computers with an underlying lattice-based qubit connectivity, such as all quantum processors based on superconducting qubits.

quant-ph↗

arXiv2022

Boundary theories of critical matchgate tensor networks

Key aspects of the AdS/CFT correspondence can be captured in terms of tensor network models on hyperbolic lattices. For tensors fulfilling the matchgate constraint, these have previously been shown to produce disordered boundary states whose site-averaged ground state properties match the translation-invariant critical Ising model. In this work, we substantially sharpen this relationship by deriving disordered local Hamiltonians generalizing the critical Ising model whose ground and low-energy excited states are accurately represented by the matchgate ansatz without any averaging. We show that these Hamiltonians exhibit multi-scale quasiperiodic symmetries captured by an analytical toy model based on layers of the hyperbolic lattice, breaking the conformal symmetries of the critical Ising model in a controlled manner. We provide a direct identification of correlation functions of ground and low-energy excited states between the disordered and translation-invariant models and give numerical evidence that the former approaches the latter in the large bond dimension limit. This establishes tensor networks on regular hyperbolic tilings as an effective tool for the study of conformal field theories. Furthermore, our numerical probes of the bulk parameters corresponding to boundary excited states constitute a first step towards a tensor network bulk-boundary dictionary between regular hyperbolic geometries and critical boundary states.

quant-ph↗

arXiv2020

On the efficacy of virtual seminars

During the SARS-CoV-2 pandemic, theoretical high-energy physics, and likely also the majority of other disciplines, are seeing a surge of virtual seminars as a primary means for scientific exchange. In this brief article, we highlight some compelling benefits of virtualizing research talks, and argue for why virtual seminars should continue even after the pandemic. Based on our extensive experience on running online talks, we also summarize some basic guidelines on organizing virtual seminars, and suggest some directions in which they could evolve.

physics.ed-ph↗