arXiv subjects

David Jansen

Publications and source records attributed to David Jansen.

16 recordsLinked to original sources

arXiv2026

Quantization-Aware Healing: A Practical Recipe for Recovering Compressed, 4-Bit LLMs

Serving large language models cheaply increasingly means shipping models that are both structurally compressed to a fraction of their parameters and quantized to 4 bits. Together these steps degrade reasoning, mathematics, coding, and long-context behavior enough to require a recovery, or healing, stage before deployment. The default recipe, quantization-aware training (QAT), re-fits the compressed, quantized model to hard labels; in our pipeline it converged slowly and collapsed past its peak. We adopted Quantization-Aware Healing (QAH) instead. Because a structurally compressed model is never independently trained at full precision, its bfloat16 checkpoint is a distillation-recovered approximation of the original; QAH distills the 4-bit student directly from the original, uncompressed model. On a GPT-OSS 120B to 60B to MXFP4 pipeline, the QAH student matches or beats its bfloat16 source on 7 of 9 benchmarks at roughly 4 times less weight memory and half the teacher's parameter count, and is released open-weight as Hypernova-60B. Against a matched QAT baseline it reaches a comparable peak about 7 times faster and stays stable under continued training, without hand-tuned early stopping. We also report deployment lessons, including a large, reproducible quality gap between distributed-training backends. Our aim is a recipe deployable without a multi-week hyper-parameter search.

cs.CL↗

arXiv2026

Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss

Small language models are often the only option for deployment under tight latency, cost, and on-premises constraints, but they are rarely trained from scratch: a compressed model is usually recovered through knowledge distillation (KD). This recovery step largely decides the final quality, yet it is expensive. We present a practitioner's study of how to make distillation training efficient, organised around two systems contributions. First, we show that offline KD (caching the teacher's top-$K$ logits once and training the student against the cache) matches online distillation at near-identical training loss while removing the teacher from memory, running about 29\% faster per iteration, and reaching up to 41\% higher throughput on a single H200 GPU. Second, we introduce a \emph{fused, chunked KL loss} that never materialises the full vocabulary-sized logit tensor, making peak memory linear in the sequence length. This removes the memory spike that otherwise caps context length and lets us train at four times the context (32{,}768 tokens) on a single GPU. A separate output-head-only toy benchmark isolates the loss kernel and confirms its memory and iteration-rate scaling from 4K to 256K tokens. Together these make large-scale healing and hundreds of ablations affordable. We also report supporting ablations on loss design and sequence packing. We release our chunked-loss implementation: https://github.com/CompactifAI/Full-Chunked-KL-Loss.

cs.CL↗

arXiv2026

Scalable Ground-State Certification of Quantum Spin Systems via Structured Noncommutative Polynomial Optimization

A fundamental challenge in quantum physics is determining the ground-state properties of many-body systems. Whereas standard variational approaches posit a wave-function ansatz and minimize over the possible states expressible by that ansatz, the problem can alternatively be formulated as a noncommutative polynomial optimization problem and treated through a hierarchy of semidefinite programming relaxations. In contrast to variational calculations, these relaxations provide lower bounds on ground-state energies and both lower and upper bounds on observable expectation values. However, this approach typically suffers from severe scalability issues, limiting its applicability to small-to-medium-scale systems. In this article, we demonstrate that systematically leveraging the inherent structures of the system can substantially mitigate these scalability challenges and thus permits computing meaningful bounds for quantum spin systems on square lattices of size up to $16\times16$.

quant-ph↗

arXiv2026

LLM Compression by Block Removal with Constrained Binary Optimization

In this paper, we formulate the compression of large language models (LLMs) by optimally deleting transformer blocks (``block removal'') as a constrained binary optimization (CBO) problem that can be mapped to a physical system (Ising glass), whose energies are a strong proxy for downstream model performance. This formulation enables an efficient ranking of a large number of candidate block-removal configurations yielding many high-quality, non-trivial solutions beyond those only removing consecutive regions. Our method performs strongly in the deep compression regime, such as for 50% compression of Llama-3.3-70B-Instruct, where we achieve an almost 23 percentage point increase on the MMLU benchmark compared to other state-of-the-art (SOTA) block-removal methods. For lighter compression, it performs on par with those methods across several benchmarks for Llama-3.1-8B-Instruct, Qwen3-14B (both before and after retraining), as well as Llama-3.3-70B-Instruct. The approach is computationally efficient and requires only forward and backward passes on a calibration dataset for a few active parameters. Additionally, we demonstrate that using good heuristic solvers for the CBO problem provides solutions that perform well on downstream tasks in negligible runtime when it is unfeasible to solve the problem exactly. The method can be readily applied to any architecture. We illustrate this generality on the recent NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 model, which exhibits a highly inhomogeneous and challenging block structure, and where we outperform SOTA for AIME25 and GPQA when removing either 2 attention layers or 3 mixture-of-experts layers.

cs.LG↗

arXiv2025

Globally optimized SVD compression of LLMs via Fermi-function-based rank selection and gauge fixing

Large Language Models (LLMs) are very demanding in terms of their computational resources. Low-rank decompositions of LLM weights, e.g. via Singular Value Decomposition (SVD), is a promising approach for LLM compression, but presents several practical hurdles, e.g. selecting appropriate layer-wise ranks and getting rid of its parameter redundancy. In this work, we present two physics-inspired improvements to SVD LLM compression: (1) \textbf{FermiGrad}, a gradient-descent algorithm that determines globally optimal layer-wise ranks by relaxing the discrete singular-value truncation into a continuous optimization using the Fermi function; (2) \textbf{PivGa}, an additional \textit{lossless} compression of the low-rank factors that exploits the intrinsic gauge freedom in their parametrization.

cs.LG↗

arXiv2025

Beyond Ground States: Physics-Inspired Optimization of Excited States of Classical Hamiltonians

We introduce excited local quantum annealing (ExcLQA), a classical, physics-inspired algorithm that extends local quantum annealing (LQA) to identify excited states of classical Ising Hamiltonians. LQA simulates quantum annealing while constraining the quantum state to remain in a product state and uses a gradient-based approach to find approximate solutions to large-scale quadratic unconstrained binary optimization problems. ExcLQA extends this framework by adding a penalty term in the cost function to target excited states, with a single hyperparameter that can be tuned via binary search to set the desired penalization level. We benchmark ExcLQA on fully connected Ising models with random interactions and on the shortest vector problem (SVP). The latter is a fundamental lattice problem underlying the security of many post-quantum cryptographic schemes, and its solution can be mapped to the first excited state of an Ising Hamiltonian. For the fully connected Ising models, we show that, on the tested instances, ExcLQA outperforms both a matrix-product-state-based method and simulated annealing. Notably, even when only a lower bound on the ground-state energy is provided, rather than the exact ground-state information required by these competing methods, ExcLQA still achieves superior performance. For the SVP, ExcLQA finds exact solutions for instances up to rank 46, and outperforms the Metropolis-Hastings algorithm in terms of solved ratio, number of shots, and approximation factor on the tested instances.

quant-ph↗

arXiv2025

Mapping Phase Diagrams of Quantum Spin Systems through Semidefinite-Programming Relaxations

Identifying quantum phase transitions poses a significant challenge in condensed matter physics, as this requires methods that both provide accurate results and scale well with system size. In this work, we demonstrate how relaxation methods can be used to generate the phase diagram for one- and two-dimensional quantum systems. To do so, we formulate a relaxed version of the ground-state problem as a semidefinite program, which we can solve efficiently. Then, by taking the resulting vector of moments for different model parameters, we identify all phase transitions based on their cosine similarity. Furthermore, we show how spontaneous symmetry breaking is naturally captured by bounding the corresponding observable. Using these methods, we reproduce the phase transitions for the one-dimensional transverse field Ising model and the two-dimensional frustrated bilayer Heisenberg model. We also illustrate how the phase diagram of the latter changes when a next-nearest-neighbor interaction is introduced. Overall, our work demonstrates how relaxation methods provide a novel framework for studying and understanding quantum phase transitions.

cond-mat.str-el↗

arXiv2024

Certifying steady-state properties of open quantum systems

Estimating the steady-state properties of open many-body quantum systems is a fundamental challenge in quantum science and technologies. In this work, we present a scalable approach based on semi-definite programming to derive certified bounds on the expectation value of an arbitrary observable in the steady state of Lindbladian dynamics. We illustrate our method on a series of many-body systems, including paradigmatic spin-1/2 chains and two-dimensional ladders, considering both equilibrium and nonequilibrium steady-states. We benchmark our method with state-of-the-art tensor-network approaches that, unlike our method, are only able to provide estimates, with no guarantee, on steady-state quantities. For the tested models, only modest computational effort is needed to obtain certified non-trivial bounds for system sizes intractable by exact methods. Our method introduces the first general numerical tool for bounding steady-state properties of open quantum dynamics, opening a new avenue in the understanding of stable configurations in many-body systems.

quant-ph↗

arXiv2024

Qudit-inspired optimization for graph coloring

We introduce a quantum-inspired algorithm for graph coloring problems (GCPs) that utilizes qudits in a product state, with each qudit representing a node in the graph and parameterized by d-dimensional spherical coordinates. We propose and benchmark two optimization strategies: qudit gradient descent, initiating qudits in random states and employing gradient descent to minimize a cost function, and qudit local quantum annealing, which adapts the local quantum annealing method to optimize an adiabatic transition from a tractable initial function to a problem-specific cost function. Our approaches are benchmarked against established solutions for standard GCPs, showing that our methods not only rival but frequently surpass the performance of recent state-of-the-art algorithms in terms of solution quality and computational efficiency. The adaptability of our algorithm and its high-quality solutions, achieved with minimal computational resources, point to an advancement in the field of quantum-inspired optimization, with potential applications extending to a broad spectrum of optimization problems.

quant-ph↗

arXiv2023

Thermal and optical conductivity in the Holstein model at half filling and at finite temperature in the Luttinger-liquid and charge-density-wave regime

Electron-phonon interactions play a key role in many branches of solid-state physics. Here, our focus is on the transport properties of one-dimensional systems, and we apply efficient real-time matrix-product state methods to compute the optical and thermal conductivities of Holstein chains at finite temperatures and filling. We validate our approach by comparison with analytical results applicable to single polarons valid in the small polaron limit. Our work provides a systematic study of contributions to the thermal conductivity at finite frequencies and elucidates differences in the spectrum compared to the optical conductivity, covering both the Luttinger-liquid and charge-density-wave regimes. Finally, we demonstrate that our approach is capable of extracting the DC conductivities as well. Beyond this first application, several future extensions seem feasible, such as, the inclusion of dispersive phonons, different types of local electron-phonon coupling, and a systematic study of drag effects in this electron-phonon coupled system.

cond-mat.str-el↗

arXiv2022

Finite-temperature optical conductivity with density-matrix renormalization group methods for the Holstein polaron and bipolaron with dispersive phonons

A comprehensive picture of polaron and bipolaron physics is essential to understand the optical absorption spectrum in many materials with electron-phonon interactions. In particular, the finite-temperature properties are of interest since they play an important role in many experiments. Here, we combine the parallel two-site time-dependent variational principle algorithm (p2TDVP) with local basis optimization (LBO) and purification to calculate time-dependent current-current correlation functions. From this information, we extract the optical conductivity for the Holstein polaron and bipolaron with dispersive phonons at finite temperatures. For the polaron in the weak and intermediate electron-phonon coupling regimes, we analyze the influence of phonon dispersion relations on the spectra. For strong electron-phonon coupling, the known result of an asymmetric Gaussian is reproduced for a flat phonon band. For a finite phonon bandwidth, the center of the Gaussian is either shifted to larger or smaller frequencies, depending on the sign of the phonon hopping. We illustrate that this can be well understood by considering the Born-Oppenheimer surfaces. A similar behavior is seen for the bipolaron for strong coupling. For the bipolaron with weak and intermediate coupling strengths and a flat phonon band, we obtain two very different spectra. The latter also has a temperature-dependent resonance at a frequency below the phonon frequency.

cond-mat.str-el↗

arXiv2022

Real-time non-adiabatic dynamics in the one-dimensional Holstein model: Trajectory-based vs exact methods

We benchmark a set of quantum-chemistry methods, including multitrajectory Ehrenfest, fewest-switches surface-hopping, and multiconfigurational-Ehrenfest dynamics, against exact quantum-many-body techniques by studying real-time dynamics in the Holstein model. This is a paradigmatic model in condensed matter theory incorporating a local coupling of electrons to Einstein phonons. For the two-site and three-site Holstein model, we discuss the exact and quantum-chemistry methods in terms of the Born-Huang formalism, covering different initial states, which either start on a single Born-Oppenheimer surface, or with the electron localized to a single site. For extended systems with up to 51 sites, we address both the physics of single Holstein polarons and the dynamics of charge-density waves at finite electron densities. For these extended systems, we compare the quantum-chemistry methods to exact dynamics obtained from time-dependent density matrix renormalization group calculations with local basis optimization (DMRG-LBO). We observe that the multitrajectory Ehrenfest method, in general, only captures the ultrashort time dynamics accurately. In contrast, the surface-hopping method with suitable corrections provides a much better description of the long-time behavior but struggles with the short-time description of coherences between different Born-Oppenheimer states. We show that the multiconfigurational Ehrenfest method yields a significant improvement over the multitrajectory Ehrenfest method and can be converged to the exact results in small systems with moderate computational efforts. We further observe that for extended systems, this convergence is slower with respect to the number of configurations. Our benchmark study demonstrates that DMRG-LBO is a useful tool for assessing the quality of the quantum-chemistry methods.

cond-mat.mtrl-sci↗

arXiv2021

Charge density wave breakdown in a heterostructure with electron-phonon coupling

Understanding the influence of vibrational degrees of freedom on transport through a heterostructure poses considerable theoretical and numerical challenges. In this work, we use the density-matrix renormalization group (DMRG) method together with local basis optimization (LBO) to study the half-filled Holstein model in the presence of a linear potential, either isolated or coupled to tight-binding leads. In both cases, we observe a decay of charge-density-wave (CDW) states at a sufficiently strong potential strength. Local basis optimization selects the most important linear combinations of local oscillator states to span the local phonon space. These states are referred to as optimal modes. We show that many of these local optimal modes are needed to capture the dynamics of the decay, that the most significant optimal mode on the initially occupied sites remains well described by a coherent-state typical for small polarons, and that those on the initially empty sites deviate from the coherent-state form. Additionally, we compute the current through the structure in the metallic regime as a function of voltage. For small voltages, we reproduce results for the Luttinger parameters. As the voltage is increased, the effect of larger electron-phonon coupling strengths becomes prominent. Further, the most significant optimal mode remains almost unchanged when going from the ground state to the current-carrying state in the metallic regime.

cond-mat.str-el↗

arXiv2020

Finite-temperature density-matrix renormalization group method for electron-phonon systems: Thermodynamics and Holstein-polaron spectral functions

We investigate the thermodynamics and finite-temperature spectral functions of the Holstein polaron using a density-matrix renormalization group method. Our method combines purification and local basis optimization (LBO) as an efficient treatment of phonon modes. LBO is a scheme which relies on finding the optimal local basis by diagonalizing the local reduced density matrix. By transforming the state into this basis, one can truncate the local Hilbert space with a negligible loss of accuracy for a wide range of parameters. In this work, we focus on the crossover regime between large and small polarons of the Holstein model. Here, no analytical solution exists and we show that the thermal expectation values at low temperatures are independent of the phonon Hilbert space truncation provided the basis is chosen large enough. We then demonstrate that we can extract the electron spectral function and establish consistency with results from a finite-temperature Lanczos method. We additionally calculate the electron emission spectrum and the phonon spectral function and show that all the computations are significantly simplified by the local basis optimization. We observe that the electron emission spectrum shifts spectral weight to both lower frequencies and larger momenta as the temperature is increased. The phonon spectral function experiences a large broadening and the polaron peak at large momenta gets significantly flattened and merges almost completely into the free-phonon peak.

cond-mat.str-el↗

arXiv2019

Eigenstate thermalization and quantum chaos in the Holstein polaron model

The eigenstate thermalization hypothesis (ETH) is a successful theory that provides sufficient criteria for ergodicity in quantum many-body systems. Most studies were carried out for Hamiltonians relevant for ultracold quantum gases and single-component systems of spins, fermions, or bosons. The paradigmatic example for thermalization in solid-state physics are phonons serving as a bath for electrons. This situation is often viewed from an open-quantum system perspective. Here, we ask whether a minimal microscopic model for electron-phonon coupling is quantum chaotic and whether it obeys ETH, if viewed as a closed quantum system. Using exact diagonalization, we address this question in the framework of the Holstein polaron model. Even though the model describes only a single itinerant electron, whose coupling to dispersionless phonons is the only integrability-breaking term, we find that the spectral statistics and the structure of Hamiltonian eigenstates exhibit essential properties of the corresponding random-matrix ensemble. Moreover, we verify the ETH ansatz both for diagonal and offdiagonal matrix elements of typical phonon and electron observables, and show that the ratio of their variances equals the value predicted from random-matrix theory.

cond-mat.str-el↗

arXiv2007

A computer program for fast non-LTE analysis of interstellar line spectra

The large quantity and high quality of modern radio and infrared line observations require efficient modeling techniques to infer physical and chemical parameters such as temperature, density, and molecular abundances. We present a computer program to calculate the intensities of atomic and molecular lines produced in a uniform medium, based on statistical equilibrium calculations involving collisional and radiative processes and including radiation from background sources. Optical depth effects are treated with an escape probability method. The program is available on the World Wide Web at http://www.sron.rug.nl/~vdtak/radex/index.shtml . The program makes use of molecular data files maintained in the Leiden Atomic and Molecular Database (LAMDA), which will continue to be improved and expanded. The performance of the program is compared with more approximate and with more sophisticated methods. An Appendix provides diagnostic plots to estimate physical parameters from line intensity ratios of commonly observed molecules. This program should form an important tool in analyzing observations from current and future radio and infrared telescopes.

astro-ph↗