arXiv subjects

Robert Seidel

Publications and source records attributed to Robert Seidel.

7 recordsLinked to original sources

arXiv2025

Choose Your Model Size: Any Compression of Large Language Models Without Re-Computation

The adoption of Foundation Models in resource-constrained environments remains challenging due to their large size and inference costs. A promising way to overcome these limitations is post-training compression, which aims to balance reduced model size against performance degradation. This work presents Any Compression via Iterative Pruning (ACIP), a novel algorithmic approach to determine a compression-performance trade-off from a single stochastic gradient descent run. To achieve parameter efficiency, we use an SVD-reparametrization of linear layers and iteratively prune their singular values with a sparsity-inducing penalty. Importantly, the pruning order of the parameters is used to derive a global score map that allows compressing a model to any target size without re-computation. We evaluate ACIP on a large selection of open-weight LLMs and downstream tasks, demonstrating state-of-the-art results compared to existing factorization-based compression methods. We also show that ACIP seamlessly complements common quantization-based compression techniques.

cs.LG↗

arXiv2021

Compressed Sensing with 1D Total Variation: Breaking Sample Complexity Barriers via Non-Uniform Recovery

This paper investigates total variation minimization in one spatial dimension for the recovery of gradient-sparse signals from undersampled Gaussian measurements. Recently established bounds for the required sampling rate state that uniform recovery of all $s$-gradient-sparse signals in $\mathbb{R}^n$ is only possible with $m \gtrsim \sqrt{s n} \cdot \text{PolyLog}(n)$ measurements. Such a condition is especially prohibitive for high-dimensional problems, where $s$ is much smaller than $n$. However, previous empirical findings seem to indicate that this sampling rate does not reflect the typical behavior of total variation minimization. The present work provides a rigorous analysis that breaks the $\sqrt{s n}$-bottleneck for a large class of "natural" signals. The main result shows that non-uniform recovery succeeds with high probability for $m \gtrsim s \cdot \text{PolyLog}(n)$ measurements if the jump discontinuities of the signal vector are sufficiently well separated. In particular, this guarantee allows for signals arising from a discretization of piecewise constant functions defined on an interval. The key ingredient of the proof is a novel upper bound for the associated conic Gaussian mean width, which is based on a signal-dependent, non-dyadic Haar wavelet transform. Furthermore, a natural extension to stable and robust recovery is addressed.

cs.IT↗

arXiv2020

Compressed Sensing with 1D Total Variation: Breaking Sample Complexity Barriers via Non-Uniform Recovery (iTWIST'20)

This paper investigates total variation minimization in one spatial dimension for the recovery of gradient-sparse signals from undersampled Gaussian measurements. Recently established bounds for the required sampling rate state that uniform recovery of all $s$-gradient-sparse signals in $\mathbb{R}^n$ is only possible with $m \gtrsim \sqrt{s n} \cdot \text{PolyLog}(n)$ measurements. Such a condition is especially prohibitive for high-dimensional problems, where $s$ is much smaller than $n$. However, previous empirical findings seem to indicate that the latter sampling rate does not reflect the typical behavior of total variation minimization. Indeed, this work provides a rigorous analysis that breaks the $\sqrt{s n}$-bottleneck for a large class of natural signals. The main result shows that non-uniform recovery succeeds with high probability for $m \gtrsim s \cdot \text{PolyLog}(n)$ measurements if the jump discontinuities of the signal vector are sufficiently well separated. In particular, this guarantee allows for signals arising from a discretization of piecewise constant functions defined on an interval. The present paper serves as a short summary of the main results in our recent work [arxiv:2001.09952].

cs.IT↗

arXiv2019

Orthogonal Matching Pursuit with Tikhonov and Landweber Regularization

The Orthogonal Matching Pursuit (OMP) for compressed sensing iterates over a scheme of support augmentation and signal estimation. We present two novel matching pursuit algorithms with intrinsic regularization of the signal estimation step that do not rely on a priori knowledge of the signal's sparsity. An iterative approach allows for a hardware efficient implementation of our algorithm, and enables real-world applications of compressed sensing. We provide a series of numerical examples that demonstrate a good performance, especially when the number of measurements is relatively small.

cs.IT↗

arXiv2015

Multi-reference approach to the calculation of photoelectron spectra including spin-orbit coupling

X-ray photoelectron spectra provide a wealth of information on the electronic structure. The extraction of molecular details requires adequate theoretical methods, which in case of transition metal complexes has to account for effects due to the multi-configurational and spin-mixed nature of the many-electron wave function. Here, the Restricted Active Space Self-Consistent Field method including spin-orbit coupling is used to cope with this challenge and to calculate valence and core photoelectron spectra. The intensities are estimated within the frameworks of the Dyson orbital formalism and the sudden approximation. Thereby, we utilize an efficient computational algorithm that is based on a biorthonormal basis transformation. The approach is applied to the valence photoionization of the gas phase water molecule and to the core ionization spectrum of the $\text{[Fe(H}_2\text{O)}_6\text{]}^{2+}$ complex. The results show good agreement with the experimental data obtained in this work, whereas the sudden approximation demonstrates distinct deviations from experiments.

physics.chem-ph↗

arXiv2004

Carbon Nanotubes for Interconnect Applications

We briefly review the status of the application of carbon nanotubes (CNTs) for future interconnects and present results concerning possible integration schemes. Growth of single nanotubes at lithographically defined locations (vias) has been achieved which is a prerequisite for the use of CNTs as future interconnects. For the 20 nm node, a current density of 5 10^8 A/cm^2 and a resistance of 7.8 kOhm could be achieved for a single multi-walled CNT vertical interconnect.

cond-mat.mtrl-sci↗

arXiv2004

Microelectronic interconnects based on carbon nanotubes

Carbon nanotubes have emerged as a possible new material for electronic applications. They show promising characteristics for transistors as well as for interconnects. Here we review their basic properties and focus on the status of nanotubes with respect to their application as interconnects and discuss the challenges facing their integration.

cond-mat.mtrl-sci↗