arXiv subjects

Philipp Grohs

Publications and source records attributed to Philipp Grohs.

At least 19 recordsLinked to original sources

arXiv2026

Is Variational Monte Carlo Robust? Sharp Moment Thresholds and Heavy-tailed Stochastic Optimization

Variational Monte Carlo (VMC) is a central algorithm in electronic structure theory and has gained renewed importance through modern neural-network ans\"atze such as FermiNet. At its core, VMC seeks ground states by minimizing the Rayleigh quotient by stochastic optimization. In this work, we show that the resulting stochastic optimization problem is intrinsically governed by the nodal geometry of the underlying wave function. More precisely, we establish that properties of the nodal set determine the integrability of the local energy and gradient estimators that drive VMC. For broad and practically relevant ansatz classes, including Slater-Jastrow wave functions with variable-exponent Slater-type orbitals, we prove that these estimators are generically heavy-tailed and fail to admit higher moments. At the same time, for general analytic ans\"atze, we prove weak moment bounds for the relevant estimators and identify precise low-moment regimes, showing how generic and degenerate nodal structures lead to different integrability thresholds. Building on this analysis, we introduce a new robust variant of VMC $\unicode{x2013}$ coined PS-Clip-VMC $\unicode{x2013}$ which is based on clipping both the local energies and the per-sample gradients. We prove that PS-Clip-VMC converges both in expectation and with high probability in the weak moment regime of VMC. The robustness of our method is confirmed experimentally by training FermiNet on atoms with up to 18 electrons.

cs.LG↗

arXiv2026

The Information-Theoretic Benefit of Shared Representations under Orthogonality Constraints

Modern deep learning architectures are increasingly multi-task and multi-modal, using a pretrained foundation model combined with task-specific, fine-tuned models. Empirically, exploiting similarity across different problems, instead of solving them individually, can significantly improve overall performance. While the generalization and sample complexity properties of multitask learning have been widely studied, the parametric complexity of joint approximation in comparison to separate approximation remains less well understood. The question is particularly relevant in modern deep learning, where models are increasingly required to satisfy structural constraints such as equivariance, conservation laws, or orthogonality. We prove lower and upper bounds on the description-length for separate and joint approximation classes, respectively, in uniform norm. We build a class of orthogonal functions by composing a shared hard feature, realized by a Rademacher-Haar wavelet series, with Sawtooth-Walsh readouts to enforce orthogonality of output coordinates. The dyadic tree structure of the Rademacher-Haar wavelet concentrates the approximation hardness in the common feature component, while the readouts act as task-specific heads. Using an information-theoretic framework, we obtain a sharp gap between the optimal approximation rates achievable by joint and separate coding. Finally, we realize this separation in a neural network model using Heaviside activations via reduction to triangle-wave approximation. Our results show that even under an orthogonality constraint joint approximation requires strictly fewer bits in compositional architectures, provided the tasks share a latent hard feature. This provides theoretical insight into the description-length-efficiency of compositional multi-output architectures and clarifies how neural networks can retain expressivity under geometric constraints.

cs.LG↗

arXiv2026

Limitations of Learning Tanh Neural Networks with Finite Precision

We investigate limitations of learning $\tanh$ neural networks from point evaluations under finite-precision computations and $L^p$ accuracy guarantees, building on Berner, Grohs, and Voigtl\"ander (2023). Our approach is based on a novel construction of sharply localized bump functions via iterated $\tanh$ activations. Using this mechanism, we show that, in a finite-precision setting, no adaptive randomized algorithm based on $m$ samples can achieve a convergence rate higher than the Monte Carlo rate $O(m^{-1/p})$ in the $L^p$ norm, unless the sampling budget grows exponentially with the size of the network parameters and architecture. The results reveal fundamental limitations imposed by finite precision on the learnability of classes containing localized bump functions, extending previous results for ReLU networks to the $\tanh$ setting.

cs.LG↗

arXiv2026

Robust and Fast Training via Per-Sample Clipping

We propose a robust gradient estimator based on per-sample gradient clipping and analyze its properties both theoretically and empirically. We show that the resulting method, per-sample clipped SGD (PS-Clip-SGD), achieves optimal in-expectation convergence rates for non-convex optimization problems under heavy-tailed gradient noise. Moreover, we establish high-probability convergence guarantees that match the in-expectation rates up to polylogarithmic factors in the failure probability. We complement our theoretical results with multiple numerical experiments. In particular, we demonstrate that PS-Clip-SGD outperforms both vanilla SGD with momentum and standard gradient clipping when training AlexNet on the CIFAR-100 dataset, even after accounting for the additional computational time caused by per-sample clipping. We also empirically show that, in the presence of gradient accumulation, applying clipping at the mini-batch level can improve training performance while incurring virtually no additional computational cost. This finding is particularly interesting, as it contradicts the common practice of applying clipping only after all accumulation steps have been completed.

math.OC↗

arXiv2025

Accurate Ab-initio Neural-network Solutions to Large-Scale Electronic Structure Problems

We present finite-range embeddings (FiRE), a novel wave function ansatz for accurate large-scale ab-initio electronic structure calculations. Compared to contemporary neural-network wave functions, FiRE reduces the asymptotic complexity of neural-network variational Monte Carlo (NN-VMC) by $\sim n_\text{el}$, the number of electrons. By restricting electron-electron interactions within the neural network, FiRE accelerates all key operations -- sampling, pseudopotentials, and Laplacian computations -- resulting in a real-world $10\times$ acceleration in now-feasible 180-electron calculations. We validate our method's accuracy on various challenging systems, including biochemical compounds, conjugated hydrocarbons, and organometallic compounds. On these systems, FiRE's energies are consistently within chemical accuracy of the most reliable data, including experiments, even in cases where high-accuracy methods such as CCSD(T), AFQMC, or contemporary NN-VMC fall short. With these improvements in both runtime and accuracy, FiRE represents a new `gold-standard' method for fast and accurate large-scale ab-initio calculations, potentially enabling new computational studies in fields like quantum chemistry, solid-state physics, and material design.

physics.comp-ph↗

arXiv2025

Theory-to-Practice Gap for Neural Networks and Neural Operators

This work studies the sampling complexity of learning with ReLU neural networks and neural operators. For mappings belonging to relevant approximation spaces, we derive upper bounds on the best-possible convergence rate of any learning algorithm, with respect to the number of samples. In the finite-dimensional case, these bounds imply a gap between the parametric and sampling complexities of learning, known as the \emph{theory-to-practice gap}. In this work, a unified treatment of the theory-to-practice gap is achieved in a general $L^p$-setting, while at the same time improving available bounds in the literature. Furthermore, based on these results the theory-to-practice gap is extended to the infinite-dimensional setting of operator learning. Our results apply to Deep Operator Networks and integral kernel-based neural operators, including the Fourier neural operator. We show that the best-possible convergence rate in a Bochner $L^p$-norm is bounded by rates of order $1/p$.

cs.LG↗

arXiv2024

The sampling complexity of learning invertible residual neural networks

In recent work it has been shown that determining a feedforward ReLU neural network to within high uniform accuracy from point samples suffers from the curse of dimensionality in terms of the number of samples needed. As a consequence, feedforward ReLU neural networks are of limited use for applications where guaranteed high uniform accuracy is required. We consider the question of whether the sampling complexity can be improved by restricting the specific neural network architecture. To this end, we investigate invertible residual neural networks which are foundational architectures in deep learning and are widely employed in models that power modern generative methods. Our main result shows that the residual neural network architecture and invertibility do not help overcome the complexity barriers encountered with simpler feedforward architectures. Specifically, we demonstrate that the computational complexity of approximating invertible residual neural networks from point samples in the uniform norm suffers from the curse of dimensionality. Similar results are established for invertible convolutional Residual neural networks.

stat.ML↗

arXiv2024

Transferable Neural Wavefunctions for Solids

Deep-Learning-based Variational Monte Carlo (DL-VMC) has recently emerged as a highly accurate approach for finding approximate solutions to the many-electron Schr\"odinger equation. Despite its favorable scaling with the number of electrons, $\mathcal{O}(n_\text{el}^{4})$, the practical value of DL-VMC is limited by the high cost of optimizing the neural network weights for every system studied. To mitigate this problem, recent research has proposed optimizing a single neural network across multiple systems, reducing the cost per system. Here we extend this approach to solids, where similar but distinct calculations using different geometries, boundary conditions, and supercell sizes are often required. We show how to optimize a single ansatz across all of these variations, reducing the required number of optimization steps by an order of magnitude. Furthermore, we exploit the transfer capabilities of a pre-trained network. We successfully transfer a network, pre-trained on 2x2x2 supercells of LiH, to 3x3x3 supercells. This reduces the number of optimization steps required to simulate the large system by a factor of 50 compared to previous work.

physics.comp-ph↗

arXiv2023

Sampling Complexity of Deep Approximation Spaces

While it is well-known that neural networks enjoy excellent approximation capabilities, it remains a big challenge to compute such approximations from point samples. Based on tools from Information-based complexity, recent work by Grohs and Voigtlaender [Journal of the FoCM (2023)] developed a rigorous framework for assessing this so-called "theory-to-practice gap". More precisely, in that work it is shown that there exist functions that can be approximated by neural networks with ReLU activation function at an arbitrary rate while requiring an exponentially growing (in the input dimension) number of samples for their numerical computation. The present study extends these findings by showing analogous results for the ReQU activation function.

cs.LG↗

arXiv2023

Phase retrieval in Fock space and perturbation of Liouville sets

We study the determination of functions in Fock space from samples of their absolute value, known as the phase retrieval problem in Fock space. An important finding in this research field asserts that phaseless sampling on lattices of arbitrary density renders the problem unsolvable. The present study establishes solvability when using irregular sampling sets of the form $A \cup B \cup C$, where $A, B,$ and $C$ constitute perturbations of a Liouville set, i.e., a set with the property that all functions in Fock space bounded on the set are constant. The sets $A, B,$ and $C$ adhere to specific geometrical conditions of closeness and noncollinearity. We show that these conditions are sufficiently generic so as to allow the perturbations to be chosen also at random. By proving that Liouville sets occupy an intermediate position between sets of stable sampling and sets of uniqueness, we obtain the first construction of uniqueness sets for the phase retrieval problem in Fock space having a finite density. The established results apply to the Gabor phase retrieval problem in subspaces of $L^2(\mathbb{R})$, where we derive additional reductions of the size of uniqueness sets: for the class of real-valued functions, uniqueness is achieved from two perturbed lattices; for the class of even real-valued functions, a single perturbation suffices, resulting in a separated set.

math.FA↗

arXiv2023

Variational Monte Carlo on a Budget -- Fine-tuning pre-trained Neural Wavefunctions

Obtaining accurate solutions to the Schr\"odinger equation is the key challenge in computational quantum chemistry. Deep-learning-based Variational Monte Carlo (DL-VMC) has recently outperformed conventional approaches in terms of accuracy, but only at large computational cost. Whereas in many domains models are trained once and subsequently applied for inference, accurate DL-VMC so far requires a full optimization for every new problem instance, consuming thousands of GPUhs even for small molecules. We instead propose a DL-VMC model which has been pre-trained using self-supervised wavefunction optimization on a large and chemically diverse set of molecules. Applying this model to new molecules without any optimization, yields wavefunctions and absolute energies that outperform established methods such as CCSD(T)-2Z. To obtain accurate relative energies, only few fine-tuning steps of this base model are required. We accomplish this with a fully end-to-end machine-learned model, consisting of an improved geometry embedding architecture and an existing SE(3)-equivariant model to represent molecular orbitals. Combining this architecture with continuous sampling of geometries, we improve zero-shot accuracy by two orders of magnitude compared to the state of the art. We extensively evaluate the accuracy, scalability and limitations of our base model on a wide variety of test systems.

physics.chem-ph↗

arXiv2023

FakET: Simulating Cryo-Electron Tomograms with Neural Style Transfer

In cryo-electron microscopy, accurate particle localization and classification are imperative. Recent deep learning solutions, though successful, require extensive training data sets. The protracted generation time of physics-based models, often employed to produce these data sets, limits their broad applicability. We introduce FakET, a method based on Neural Style Transfer, capable of simulating the forward operator of any cryo transmission electron microscope. It can be used to adapt a synthetic training data set according to reference data producing high-quality simulated micrographs or tilt-series. To assess the quality of our generated data, we used it to train a state-of-the-art localization and classification architecture and compared its performance with a counterpart trained on benchmark data. Remarkably, our technique matches the performance, boosts data generation speed 750 times, uses 33 times less memory, and scales well to typical transmission electron microscope detector sizes. It leverages GPU acceleration and parallel processing. The source code is available at https://github.com/paloha/faket.

cs.LG↗

arXiv2023

Towards a Foundation Model for Neural Network Wavefunctions

Deep neural networks have become a highly accurate and powerful wavefunction ansatz in combination with variational Monte Carlo methods for solving the electronic Schr\"odinger equation. However, despite their success and favorable scaling, these methods are still computationally too costly for wide adoption. A significant obstacle is the requirement to optimize the wavefunction from scratch for each new system, thus requiring long optimization. In this work, we propose a novel neural network ansatz, which effectively maps uncorrelated, computationally cheap Hartree-Fock orbitals, to correlated, high-accuracy neural network orbitals. This ansatz is inherently capable of learning a single wavefunction across multiple compounds and geometries, as we demonstrate by successfully transferring a wavefunction model pre-trained on smaller fragments to larger compounds. Furthermore, we provide ample experimental evidence to support the idea that extensive pre-training of a such a generalized wavefunction model across different compounds and geometries could lead to a foundation wavefunction model. Such a model could yield high-accuracy ab-initio energies using only minimal computational effort for fine-tuning and evaluation of observables.

physics.comp-ph↗

arXiv2022

From completeness of discrete translates to phaseless sampling of the short-time Fourier transform

We study the uniqueness problem in short-time Fourier transform phase retrieval by exploring a connection to the completeness problem of discrete translates. Specifically, we prove that functions in $L^2(K)$ with $K \subseteq \mathbb{R}^d$ compact, are uniquely determined by phaseless lattice-samples of its short-time Fourier transform with window function $g$, provided that specific density properties of translates of $g$ are met. By proving completeness statements for systems of discrete translates in Banach function spaces on compact sets, we obtain new uniqueness statements for phaseless sampling on lattices beyond the known Gaussian window regime. Our results apply to a large class of window functions, which are relevant in time-frequency analysis and applications.

math.FA↗

arXiv2022

Phaseless sampling on square-root lattices

Due to its appearance in a remarkably wide field of applications, such as audio processing and coherent diffraction imaging, the short-time Fourier transform (STFT) phase retrieval problem has seen a great deal of attention in recent years. A central problem in STFT phase retrieval concerns the question for which window functions $g \in L^2(\mathbb{R}^d)$ and which sampling sets $\Lambda \subseteq \mathbb{R}^{2d}$ is every $f \in L^2(\mathbb{R}^d)$ uniquely determined (up to a global phase factor) by phaseless samples of the form $$ |V_gf(\Lambda)| = \left \{ |V_gf(\lambda)| : \lambda \in \Lambda \right \}, $$ where $V_gf$ denotes the short-time Fourier transform (STFT) of $f$ with respect to $g$. The investigation of this question constitutes a key step towards making the problem computationally tractable. However, it deviates from ordinary sampling tasks in a fundamental and subtle manner: recent results demonstrate that uniqueness is unachievable if $\Lambda$ is a lattice, i.e $\Lambda = A\mathbb{Z}^{2d}, A \in \mathrm{GL}(2d,\mathbb{R})$. Driven by this discretization barrier, the present article centers around the initiation of a novel sampling scheme which allows for unique recovery of any square-integrable function via phaseless STFT-sampling. Specifically, we show that square-root lattices, i.e., sets of the form $$ \Lambda = A \left ( \sqrt{\mathbb{Z}} \right )^{2d}, \ \sqrt{\mathbb{Z}} = \{ \pm \sqrt{n} : n \in \mathbb{N}_0 \}, $$ guarantee uniqueness of the STFT phase retrieval problem. The result holds for a large class of window functions, including Gaussians.

math.FA↗

arXiv2022

Multi-window STFT phase retrieval: lattice uniqueness

Short-time Fourier transform (STFT) phase retrieval refers to the reconstruction of a function $f$ from its spectrogram, i.e., the magnitudes of its short-time Fourier transform $V_gf$ with window function $g$. While it is known that for appropriate windows, any function $f \in L^2(\mathbb{R})$ can be reconstructed from the full spectrogram $|V_g f(\mathbb{R}^2)|$, in practical scenarios, the reconstruction must be achieved from discrete samples, typically taken on a lattice. It turns out that the sampled problem becomes much more subtle: recent results have demonstrated that uniqueness via lattice-sampling is unachievable, irrespective of the choice of the window function or the lattice density. In the present paper, we initiate the study of multi-window STFT phase retrieval as a way to effectively bypass the discretization barriers encountered in the single-window case. By establishing a link between multi-window Gabor systems, sampling in Fock space, and phase retrieval for finite frames, we derive conditions under which square-integrable functions can be uniquely recovered from spectrogram samples on a lattice. Specifically, we provide conditions on window functions $g_1, \dots, g_4 \in L^2(\mathbb{R})$, such that every $f \in L^2(\mathbb{R})$ is determined up to a global phase from $$\left(|V_{g_1}f(A\mathbb{Z}^2)|, \, \dots, \, |V_{g_4}f(A\mathbb{Z}^2)| \right)$$ whenever $A \in \mathrm{GL}_2(\mathbb{R})$ satisfies the density condition $|\det A|^{-1} \geq 4$. For real-valued functions, a density of $|\det A|^{-1} \geq 2$ is sufficient. Corresponding results for irregular sampling are also shown.

math.FA↗

arXiv2022

Non-uniqueness theory in sampled STFT phase retrieval

The reconstruction of a function from its spectrogram (i.e., the absolute value of its short-time Fourier transform (STFT)) arises as a key problem in several important applications, including coherent diffraction imaging and audio processing. It is a classical result that for suitable windows any function can, in principle, be uniquely recovered up to a global phase factor from its spectrogram. However, for most practical applications only discrete samples - typically from a lattice - of the spectrogram are available. This raises the question of whether lattice samples of the spectrogram contain sufficient information for determining a function $f\in L^2(\mathbb{R}^d)$ up to a global phase factor. In the present paper, we answer this question in the negative by providing general non-identifiability results which lead to a non-uniqueness theory for the sampled STFT phase retrieval problem. Precisely, given any dimension $d$, any window function $g$ and any (symplectic or separable) lattice $\mathcal{L} \subseteq \mathbb{R}^d$, we construct pairs of functions $f,h\in L^2(\mathbb{R}^d)$ that do not agree up to a global phase factor, but whose spectrograms agree on $\mathcal{L}$. Our techniques are sufficiently flexible to produce counterexamples to unique recoverability under even more stringent assumptions; for example, if the window function is real-valued, the functions $f,h$ can even be chosen to satisfy $|f|=|h|$. Our results thus reveal the non-existence of a critical sampling density in the absence of phase information, a property which is in stark contrast to uniqueness results in time-frequency analysis.

math.FA↗

arXiv2022

Learning ReLU networks to high uniform accuracy is intractable

Statistical learning theory provides bounds on the necessary number of training samples needed to reach a prescribed accuracy in a learning problem formulated over a given target class. This accuracy is typically measured in terms of a generalization error, that is, an expected value of a given loss function. However, for several applications -- for example in a security-critical context or for problems in the computational sciences -- accuracy in this sense is not sufficient. In such cases, one would like to have guarantees for high accuracy on every input value, that is, with respect to the uniform norm. In this paper we precisely quantify the number of training samples needed for any conceivable training algorithm to guarantee a given uniform accuracy on any learning problem formulated over target classes containing (or consisting of) ReLU neural networks of a prescribed architecture. We prove that, under very general assumptions, the minimal number of training samples for this task scales exponentially both in the depth and the input dimension of the network architecture.

cs.LG↗