arXiv subjects

Frank Schäfer

Publications and source records attributed to Frank Schäfer.

At least 19 recordsLinked to original sources

arXiv2026

A Blueprint for Equilibrium-Based Differentiable Continuous-Variable Thermodynamic Computing

To help address the escalating energy and latency demands of machine-learning workloads, we introduce a blueprint for an energy-efficient and fast thermodynamic computing stack that leverages stochastic analog processes in physical hardware. In this work, we focus on energy-based thermodynamic computing where the stochastic process is well described by Langevin dynamics with tunable energy potentials. The implementation of such potentials in physical hardware enables us to generate and sample from basic parameterized energy-based models. We demonstrate how to construct and train popular classes of machine learning models based on these hardware-native energy-based models, using the framework of probabilistic graphical models. We analyze the runtime and energy consumption of different models in this thermodynamic paradigm based on theoretical considerations and numerical studies. As a preliminary experimental realization of such hardware, we present our stochastic analog superconducting circuits driven by thermal noise. Together, these results outline a path toward energy-efficient thermodynamic hardware for probabilistic machine learning.

cs.LG↗

arXiv2025

Differentiable Programming for Differential Equations: A Review

The differentiable programming paradigm is a cornerstone of modern scientific computing. It refers to numerical methods for computing the gradient of a numerical model's output. Many scientific models are based on differential equations, where differentiable programming plays a crucial role in calculating model sensitivities, inverting model parameters, and training hybrid models that combine differential equations with data-driven approaches. Furthermore, recognizing the strong synergies between inverse methods and machine learning offers the opportunity to establish a coherent framework applicable to both fields. Differentiating functions based on the numerical solution of differential equations is non-trivial. Numerous methods based on a wide variety of paradigms have been proposed in the literature, each with pros and cons specific to the type of problem investigated. Here, we provide a comprehensive review of existing techniques to compute derivatives of numerical solutions of differential equations. We first discuss the importance of gradients of solutions of differential equations in a variety of scientific domains. Second, we lay out the mathematical foundations of the various approaches and compare them with each other. Third, we cover the computational considerations and explore the solutions available in modern scientific software. Last but not least, we provide best-practices and recommendations for practitioners. We hope that this work accelerates the fusion of scientific models and data, and fosters a modern approach to scientific modelling.

math.NA↗

arXiv2024

Performance Bounds for Quantum Feedback Control

The limits of quantum feedback control have immediate consequences for quantum information science at large, yet remain largely unexplored. Here, we combine quantum filtering theory and moment-sum-of-squares techniques to construct a hierarchy of convex optimization problems that furnish monotonically improving, computable bounds on the best attainable performance for a broad class of quantum feedback control problems. These bounds may serve as witnesses of fundamental limitations, optimality certificates, or performance targets. We prove convergence of the bounds to the optimal control performance under technical conditions and demonstrate the practical utility of our approach by designing certifiably near-optimal controllers for a qubit in a cavity subjected to photon counting and homodyne detection measurements.

quant-ph↗

arXiv2024

Machine learning the Ising transition: A comparison between discriminative and generative approaches

The detection of phase transitions is a central task in many-body physics. To automate this process, the task can be phrased as a classification problem. Classification problems can be approached in two fundamentally distinct ways: through either a discriminative or a generative method. In general, it is unclear which of these two approaches is most suitable for a given problem. The choice is expected to depend on factors such as the availability of system knowledge, dataset size, desired accuracy, computational resources, and other considerations. In this work, we answer the question of how one should approach the solution of phase-classification problems by performing a numerical case study on the thermal phase transition in the classical two-dimensional square-lattice ferromagnetic Ising model.

cond-mat.dis-nn↗

arXiv2024

Phase Transitions in the Output Distribution of Large Language Models

In a physical system, changing parameters such as temperature can induce a phase transition: an abrupt change from one state of matter to another. Analogous phenomena have recently been observed in large language models. Typically, the task of identifying phase transitions requires human analysis and some prior understanding of the system to narrow down which low-dimensional properties to monitor and analyze. Statistical methods for the automated detection of phase transitions from data have recently been proposed within the physics community. These methods are largely system agnostic and, as shown here, can be adapted to study the behavior of large language models. In particular, we quantify distributional changes in the generated output via statistical distances, which can be efficiently estimated with access to the probability distribution over next-tokens. This versatile approach is capable of discovering new phases of behavior and unexplored transitions -- an ability that is particularly exciting in light of the rapid development of language models and their emergent capabilities.

cs.LG↗

arXiv2023

Machine learning phase transitions: Connections to the Fisher information

Despite the widespread use and success of machine-learning techniques for detecting phase transitions from data, their working principle and fundamental limits remain elusive. Here, we explain the inner workings and identify potential failure modes of these techniques by rooting popular machine-learning indicators of phase transitions in information-theoretic concepts. Using tools from information geometry, we prove that several machine-learning indicators of phase transitions approximate the square root of the system's (quantum) Fisher information from below -- a quantity that is known to indicate phase transitions but is often difficult to compute from data. We numerically demonstrate the quality of these bounds for phase transitions in classical and quantum systems.

cond-mat.dis-nn↗

arXiv2023

Fast Detection of Phase Transitions with Multi-Task Learning-by-Confusion

Machine learning has been successfully used to study phase transitions. One of the most popular approaches to identifying critical points from data without prior knowledge of the underlying phases is the learning-by-confusion scheme. As input, it requires system samples drawn from a grid of the parameter whose change is associated with potential phase transitions. Up to now, the scheme required training a distinct binary classifier for each possible splitting of the grid into two sides, resulting in a computational cost that scales linearly with the number of grid points. In this work, we propose and showcase an alternative implementation that only requires the training of a single multi-class classifier. Ideally, such multi-task learning eliminates the scaling with respect to the number of grid points. In applications to the Ising model and an image dataset generated with Stable Diffusion, we find significant speedups that closely correspond to the ideal case, with only minor deviations.

cs.LG↗

arXiv2023

Mapping out phase diagrams with generative classifiers

One of the central tasks in many-body physics is the determination of phase diagrams. However, mapping out a phase diagram generally requires a great deal of human intuition and understanding. To automate this process, one can frame it as a classification task. Typically, classification problems are tackled using discriminative classifiers that explicitly model the probability of the labels for a given sample. Here we show that phase-classification problems are naturally suitable to be solved using generative classifiers based on probabilistic models of the measurement statistics underlying the physical system. Such a generative approach benefits from modeling concepts native to the realm of statistical and quantum physics, as well as recent advances in machine learning. This leads to a powerful framework for the autonomous determination of phase diagrams with little to no human supervision that we showcase in applications to classical equilibrium systems and quantum ground states.

quant-ph↗

arXiv2023

Differentiating Metropolis-Hastings to Optimize Intractable Densities

We develop an algorithm for automatic differentiation of Metropolis-Hastings samplers, allowing us to differentiate through probabilistic inference, even if the model has discrete components within it. Our approach fuses recent advances in stochastic automatic differentiation with traditional Markov chain coupling schemes, providing an unbiased and low-variance gradient estimator. This allows us to apply gradient-based optimization to objectives expressed as expectations over intractable target densities. We demonstrate our approach by finding an ambiguous observation in a Gaussian mixture model and by maximizing the specific heat in an Ising model.

stat.ML↗

arXiv2023

Automatic Differentiation of Programs with Discrete Randomness

Automatic differentiation (AD), a technique for constructing new programs which compute the derivative of an original program, has become ubiquitous throughout scientific computing and deep learning due to the improved performance afforded by gradient-based optimization. However, AD systems have been restricted to the subset of programs that have a continuous dependence on parameters. Programs that have discrete stochastic behaviors governed by distribution parameters, such as flipping a coin with probability $p$ of being heads, pose a challenge to these systems because the connection between the result (heads vs tails) and the parameters ($p$) is fundamentally discrete. In this paper we develop a new reparameterization-based methodology that allows for generating programs whose expectation is the derivative of the expectation of the original program. We showcase how this method gives an unbiased and low-variance estimator which is as automated as traditional AD mechanisms. We demonstrate unbiased forward-mode AD of discrete-time Markov chains, agent-based models such as Conway's Game of Life, and unbiased reverse-mode AD of a particle filter. Our code package is available at https://github.com/gaurav-arya/StochasticAD.jl.

cs.LG↗

arXiv2022

Automated reconstruction of bound states in bilayer graphene quantum dots

Bilayer graphene is a nanomaterial that allows for well-defined, separated quantum states to be defined by electrostatic gating and, therefore, provides an attractive platform to construct tunable quantum dots. When a magnetic field perpendicular to the graphene layers is applied, the graphene valley degeneracy is lifted, and splitting of the energy levels of the dot is observed. Given the experimental ability to engineer this energy valley splitting, bilayer graphene quantum dots have a great potential for hosting robust qubits. Although bilayer graphene quantum dots have been recently realized in experiments, it is critically important to devise robust methods that can identify the observed quantum states from accessible measurement data. Here, we develop an efficient algorithm for extracting the model parameters needed to characterize the states of a bilayer graphene quantum dot completely. We introduce a Hamiltonian-guided random search method and demonstrate robust identification of quantum states on both simulated and experimental data.

cond-mat.mes-hall↗

arXiv2022

Data-driven reconstruction of spectral conductivity and chemical potential from thermoelectric transport data

The spectral conductivity, i.e., the electrical conductivity as a function of the Fermi energy, is a cornerstone in determining the thermoelectric transport properties of electrons. However, the spectral conductivity depends on sample-specific properties such as carrier concentrations, vacancies, charge impurities, chemical compositions, and material microstructures, making it difficult to relate the experimental result with the theoretical prediction directly. Here, we propose a data-driven approach based on machine learning to reconstruct the spectral conductivity and chemical potential from the thermoelectric transport data. Using this machine learning method, we first demonstrate that the spectral conductivity and temperature-dependent chemical potentials can be recovered within a simple toy model. In a second step, we apply our method to experimental data in doped one-dimensional telluride Ta$_4$SiTe$_4$~[T. Inohara, \textit{et al.}, Appl. Phys. Lett. \textbf{110}, 183901 (2017)] to reconstruct the spectral conductivity and chemical potential for each sample. Furthermore, the thermal conductivity of electrons and the maximal figure of merit $ZT$ are estimated from the reconstructed spectral conductivity, which provides accurate estimates beyond the Wiedemann-Franz law. Our study clarifies the connection between the thermoelectric transport properties and the low-energy electronic states of real materials, and establishes a promising route to incorporate experimental data into traditional theory-driven workflows.

cond-mat.stat-mech↗

arXiv2022

Replacing neural networks by optimal analytical predictors for the detection of phase transitions

Identifying phase transitions and classifying phases of matter is central to understanding the properties and behavior of a broad range of material systems. In recent years, machine-learning (ML) techniques have been successfully applied to perform such tasks in a data-driven manner. However, the success of this approach notwithstanding, we still lack a clear understanding of ML methods for detecting phase transitions, particularly of those that utilize neural networks (NNs). In this work, we derive analytical expressions for the optimal output of three widely used NN-based methods for detecting phase transitions. These optimal predictions correspond to the results obtained in the limit of high model capacity. Therefore, in practice they can, for example, be recovered using sufficiently large, well-trained NNs. The inner workings of the considered methods are revealed through the explicit dependence of the optimal output on the input data. By evaluating the analytical expressions, we can identify phase transitions directly from experimentally accessible data without training NNs, which makes this procedure favorable in terms of computation time. Our theoretical results are supported by extensive numerical simulations covering, e.g., topological, quantum, and many-body localization phase transitions. We expect similar analyses to provide a deeper understanding of other classification tasks in condensed matter physics.

cond-mat.dis-nn↗

arXiv2022

AbstractDifferentiation.jl: Backend-Agnostic Differentiable Programming in Julia

No single Automatic Differentiation (AD) system is the optimal choice for all problems. This means informed selection of an AD system and combinations can be a problem-specific variable that can greatly impact performance. In the Julia programming language, the major AD systems target the same input and thus in theory can compose. Hitherto, switching between AD packages in the Julia Language required end-users to familiarize themselves with the user-facing API of the respective packages. Furthermore, implementing a new, usable AD package required AD package developers to write boilerplate code to define convenience API functions for end-users. As a response to these issues, we present AbstractDifferentiation.jl for the automatized generation of an extensive, unified, user-facing API for any AD package. By splitting the complexity between AD users and AD developers, AD package developers only need to implement one or two primitive definitions to support various utilities for AD users like Jacobians, Hessians and lazy product operators from native primitives such as pullbacks or pushforwards, thus removing tedious -- but so far inevitable -- boilerplate code, and enabling the easy switching and composing between AD implementations for end-users.

cs.MS↗

arXiv2021

Control of Stochastic Quantum Dynamics by Differentiable Programming

Control of the stochastic dynamics of a quantum system is indispensable in fields such as quantum information processing and metrology. However, there is no general ready-made approach to the design of efficient control strategies. Here, we propose a framework for the automated design of control schemes based on differentiable programming ($\partial \mathrm{P}$). We apply this approach to the state preparation and stabilization of a qubit subjected to homodyne detection. To this end, we formulate the control task as an optimization problem where the loss function quantifies the distance from the target state, and we employ neural networks (NNs) as controllers. The system's time evolution is governed by a stochastic differential equation (SDE). To implement efficient training, we backpropagate the gradient information from the loss function through the SDE solver using adjoint sensitivity methods. As a first example, we feed the quantum state to the controller and focus on different methods of obtaining gradients. As a second example, we directly feed the homodyne detection signal to the controller. The instantaneous value of the homodyne current contains only very limited information on the actual state of the system, masked by unavoidable photon-number fluctuations. Despite the resulting poor signal-to-noise ratio, we can train our controller to prepare and stabilize the qubit to a target state with a mean fidelity of around 85%. We also compare the solutions found by the NN to a hand-crafted control strategy.

quant-ph↗

arXiv2020

Interpretable and unsupervised phase classification

Fully automated classification methods that yield direct physical insights into phase diagrams are of current interest. Here, we demonstrate an unsupervised machine learning method for phase classification which is rendered interpretable via an analytical derivation of its optimal predictions and allows for an automated construction scheme for order parameters. Based on these findings, we propose and apply an alternative, physically-motivated, data-driven scheme which relies on the difference between mean input features. This mean-based method is computationally cheap and directly interpretable. As an example, we consider the physically rich ground-state phase diagram of the spinless Falicov-Kimball model.

cond-mat.dis-nn↗

arXiv2020

A differentiable programming method for quantum control

Optimal control is highly desirable in many current quantum systems, especially to realize tasks in quantum information processing. We introduce a method based on differentiable programming to leverage explicit knowledge of the differential equations governing the dynamics of the system. In particular, a control agent is represented as a neural network that maps the state of the system at a given time to a control pulse. The parameters of this agent are optimized via gradient information obtained by direct differentiation through both the neural network \emph{and} the differential equation of the system. This fully differentiable reinforcement learning approach ultimately yields time-dependent control parameters optimizing a desired figure of merit. We demonstrate the method's viability and robustness to noise in eigenstate preparation tasks for three systems: a~single qubit, a~chain of qubits, and a quantum parametric oscillator.

quant-ph↗

arXiv2020

Spectral Structure and Many-Body Dynamics of Ultracold Bosons in a Double-Well

We examine the spectral structure and many-body dynamics of two and three repulsively interacting bosons trapped in a one-dimensional double-well, for variable barrier height, inter-particle interaction strength, and initial conditions. By exact diagonalization of the many-particle Hamiltonian, we specifically explore the dynamical behaviour of the particles launched either at the single particle ground state or saddle point energy, in a time-independent potential. We complement these results by a characterisation of the cross-over from diabatic to quasi-adiabatic evolution under finite-time switching of the potential barrier, via the associated time-evolution of a single particle's von Neumann entropy. This is achieved with the help of the multiconfigurational time-dependent Hartree method for indistinguishable particles (\textsc{Mctdh-x}) -- which also allows us to extrapolate our results for increasing particle numbers.

quant-ph↗