Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2608.30741

Functional Degeneracy in Neural Networks: Measurement and Pruning

Maria Matveev·Pascal Esser·Ayush Bharadwaj·Lucius Bushnaq·Gitta Kutyniok
View original source ↗Read source document ↗Export metadata ↓

Abstract

A central question in modern machine learning is how much a trained model can be compressed without changing its behavior, to reduce the memory, compute and energy required to deploy it. To study this, we quantify functional degeneracy through the behavioral recovery rank, defined as the number of leading behavioral-Hessian eigendirections required to recover a trained model's performance. Using the behavioral recovery rank as a geometric benchmark for compression, we find that structural and magnitude pruning retain more degrees of freedom, even after the task is saturated. This gap suggests that functional redundancy is distributed across parameter directions and is not exposed by individual weights or neurons.

Explore related subjects

cs.LG

Keep this discovery

BibTeXRIS

Maria Matveev, Pascal Esser, Ayush Bharadwaj, Lucius Bushnaq, Gitta Kutyniok. 2026-08-31. Functional Degeneracy in Neural Networks: Measurement and Pruning. https://arxiv.org/abs/2608.30741

Cite the original work for its findings. Save a collection to share your selection of sources.

Discover connections

cs.LG subject

Connections use source metadata and explicit phrase matches, not verified experimental comparisons.

KEEP EXPLORING

Related papers

arXiv2026

Higher Structures in Deep Learning

We provide an expository introduction on the importance of higher-arity tensor operations to deep learning. Then, we conduct a novel empirical investigation of higher-arity phenomenon in trained neural networks, introduce a hypergraphical generalization of the multilayer perceptron, and explore connections to evolutionary algorithms. We conclude with a discussion of promising directions for future research.

cs.LG↗
arXiv2026

One-Layer Transformer Provably Learns Multiclass One-Nearest Neighbor in Context

We extend recent work establishing an equivalence between one-layer transformers and nearest-neighbor classifiers in the binary setting to the multiclass case. By leveraging the simplex encoding, we show that one-layer transformers with an argmax classification head behave identically to a one-nearest-neighbor classifier in the multiclass setting. This closes a gap left by prior work, whose multiclass result relied on a non-standard rounding-based approach rather than the typical argmax head used in practice.

cs.LG↗