arXiv subjects

Florian Schäfer

Publications and source records attributed to Florian Schäfer.

At least 37 records · Page 2Linked to original sources

arXiv2023

Sparse recovery of elliptic solvers from matrix-vector products

In this work, we show that solvers of elliptic boundary value problems in $d$ dimensions can be approximated to accuracy $ε$ from only $\mathcal{O}\left(\log(N)\log^{d}(N / ε)\right)$ matrix-vector products with carefully chosen vectors (right-hand sides). The solver is only accessed as a black box, and the underlying operator may be unknown and of an arbitrarily high order. Our algorithm (1) has complexity $\mathcal{O}\left(N\log^2(N)\log^{2d}(N / ε)\right)$ and represents the solution operator as a sparse Cholesky factorization with $\mathcal{O}\left(N\log(N)\log^{d}(N / ε)\right)$ nonzero entries, (2) allows for embarrassingly parallel evaluation of the solution operator and the computation of its log-determinant, (3) allows for $\mathcal{O}\left(\log(N)\log^{d}(N / ε)\right)$ complexity computation of individual entries of the matrix representation of the solver that, in turn, enables its recompression to an $\mathcal{O}\left(N\log^{d}(N / ε)\right)$ complexity representation. As a byproduct, our compression scheme produces a homogenized solution operator with near-optimal approximation accuracy. By polynomial approximation, we can also approximate the continuous Green's function (in operator and Hilbert-Schmidt norm) to accuracy $ε$ from $\mathcal{O}\left(\log^{1 + d}\left(ε^{-1}\right)\right)$ solutions of the PDE. We include rigorous proofs of these results. To the best of our knowledge, our algorithm achieves the best known trade-off between accuracy $ε$ and the number of required matrix-vector products.

math.NA↗

arXiv2023

Fast Macroscopic Forcing Method

The macroscopic forcing method (MFM) of Mani and Park and similar methods for obtaining turbulence closure operators, such as the Green's function-based approach of Hamba, recover reduced solution operators from repeated direct numerical simulations (DNS). MFM has been used to quantify RANS-like operators for homogeneous isotropic turbulence and turbulent channel flows. Standard algorithms for MFM force each coarse-scale degree of freedom (i.e., degree of freedom in the RANS space) and conduct a corresponding fine-scale simulation (i.e., DNS), which is expensive. We combine this method with an approach recently proposed by Schäfer and Owhadi (2023) to recover elliptic integral operators from a polylogarithmic number of matrix-vector products. The resulting Fast MFM introduced in this work applies sparse reconstruction to expose local features in the closure operator and reconstructs this coarse-grained differential operator in only a few matrix-vector products and correspondingly, a few MFM simulations. For flows with significant nonlocality, the algorithm first "peels" long-range effects with dense matrix-vector products to expose a local operator. We demonstrate the algorithm's performance for scalar transport in a laminar channel flow and momentum transport in a turbulent one. For these, we recover eddy diffusivity operators at 1% of the cost of computing the exact operator via a brute-force approach for the laminar channel flow problem and 13% for the turbulent one. We observe that we can reconstruct these operators with an increase in accuracy by about a factor of 100 over randomized low-rank methods. We glean that for problems in which the RANS space is reducible to one dimension, eddy diffusivity and eddy viscosity operators can be reconstructed with reasonable accuracy using only a few simulations, regardless of simulation resolution or degrees of freedom.

physics.comp-ph↗

arXiv2023

Realization of a quantum degenerate mixture of highly magnetic and nonmagnetic atoms

We report on the experimental realization of a bosonic quantum degenerate mixture of highly-magnetic 168Er and nonmagnetic 174Yb. Quantum degeneracy is reached by forced evaporation in an all-optical trap. Formation of the two Bose-Einstein condensates is confirmed by analysis of the cloud shape and the observed inversions of the aspect ratios. The results open a path for possible new experiments on magnetic and nonmagnetic impurity physics as well as on the quantum chaotic behavior of Feshbach resonances and their dependencies on minor variations of the reduced masses.

cond-mat.quant-gas↗

arXiv2023

Observation of Feshbach resonances in an ${}^{167}$Er-${}^6$Li Fermi-Fermi mixture

We present our experimental investigation of the interspecies Feshbach spectrum in a mixture of ${}^{167}$Er($F = 19/2, m_F = -19/2$)-${}^6$Li($F = 1/2, m_F = 1/2$) atoms in the microkelvin temperature regime. These temperatures are achieved by means of sympathetic cooling with ${}^{174}$Yb as a third species. Interspecies Feshbach resonances are then identified by investigation of the Er-Li inelastic collisional properties for magnetic fields up to 800 G. Numerous narrow resonances as well as six resonances with widths above 1 G could be identified. It is these broader resonances that hold much promise for interesting future investigations of, for exmample, novel superfluid states and Efimov states in large mass-imbalanced, all-fermionic two-component systems.

cond-mat.quant-gas↗

arXiv2023

Scalable Bayesian transport maps for high-dimensional non-Gaussian spatial fields

A multivariate distribution can be described by a triangular transport map from the target distribution to a simple reference distribution. We propose Bayesian nonparametric inference on the transport map by modeling its components using Gaussian processes. This enables regularization and uncertainty quantification of the map estimation, while still resulting in a closed-form and invertible posterior map. We then focus on inferring the distribution of a nonstationary spatial field from a small number of replicates. We develop specific transport-map priors that are highly flexible and are motivated by the behavior of a large class of stochastic processes. Our approach is scalable to high-dimensional distributions due to data-dependent sparsity and parallel computations. We also discuss extensions, including Dirichlet process mixtures for flexible marginals. We present numerical results to demonstrate the accuracy, scalability, and usefulness of our methods, including statistical emulation of non-Gaussian climate-model output.

stat.ME↗

arXiv2022

ZerO Initialization: Initializing Neural Networks with only Zeros and Ones

Deep neural networks are usually initialized with random weights, with adequately selected initial variance to ensure stable signal propagation during training. However, selecting the appropriate variance becomes challenging especially as the number of layers grows. In this work, we replace random weight initialization with a fully deterministic initialization scheme, viz., ZerO, which initializes the weights of networks with only zeros and ones (up to a normalization factor), based on identity and Hadamard transforms. Through both theoretical and empirical studies, we demonstrate that ZerO is able to train networks without damaging their expressivity. Applying ZerO on ResNet achieves state-of-the-art performance on various datasets, including ImageNet, which suggests random weights may be unnecessary for network initialization. In addition, ZerO has many benefits, such as training ultra deep networks (without batch-normalization), exhibiting low-rank learning trajectories that result in low-rank and sparse solutions, and improving training reproducibility.

cs.LG↗

arXiv2022

Competitive Physics Informed Networks

Neural networks can be trained to solve partial differential equations (PDEs) by using the PDE residual as the loss function. This strategy is called "physics-informed neural networks" (PINNs), but it currently cannot produce high-accuracy solutions, typically attaining about $0.1\%$ relative error. We present an adversarial approach that overcomes this limitation, which we call competitive PINNs (CPINNs). CPINNs train a discriminator that is rewarded for predicting mistakes the PINN makes. The discriminator and PINN participate in a zero-sum game with the exact PDE solution as an optimal strategy. This approach avoids squaring the large condition numbers of PDE discretizations, which is the likely reason for failures of previous attempts to decrease PINN errors even on benign problems. Numerical experiments on a Poisson problem show that CPINNs achieve errors four orders of magnitude smaller than the best-performing PINN. We observe relative errors on the order of single-precision accuracy, consistently decreasing with each epoch. To the authors' knowledge, this is the first time this level of accuracy and convergence behavior has been achieved. Additional experiments on the nonlinear Schrödinger, Burgers', and Allen-Cahn equation show that the benefits of CPINNs are not limited to linear problems.

cs.LG↗

arXiv2021

Polymatrix Competitive Gradient Descent

Many economic games and machine learning approaches can be cast as competitive optimization problems where multiple agents are minimizing their respective objective function, which depends on all agents' actions. While gradient descent is a reliable basic workhorse for single-agent optimization, it often leads to oscillation in competitive optimization. In this work we propose polymatrix competitive gradient descent (PCGD) as a method for solving general sum competitive optimization involving arbitrary numbers of agents. The updates of our method are obtained as the Nash equilibria of a local polymatrix approximation with a quadratic regularization, and can be computed efficiently by solving a linear system of equations. We prove local convergence of PCGD to stable fixed points for $n$-player general-sum games, and show that it does not require adapting the step size to the strength of the player-interactions. We use PCGD to optimize policies in multi-agent reinforcement learning and demonstrate its advantages in Snake, Markov soccer and an electricity market game. Agents trained by PCGD outperform agents trained with simultaneous gradient descent, symplectic gradient adjustment, and extragradient in Snake and Markov soccer games and on the electricity market game, PCGD trains faster than both simultaneous gradient descent and the extragradient method.

cs.LG↗

arXiv2021

Feshbach resonances of large mass-imbalance Er-Li mixtures

We report on the experimental observation of Feshbach resonances in large mass-imbalance mixtures of Erbium (Er) and Lithium (Li). All combinations between ${}^{168}$Er, ${}^{166}$Er and ${}^7$Li, ${}^6$Li are cooled to temperatures of a few microkelvin, partially by means of sympathetic cooling together with Ytterbium (Yb) as a third mixture component. The Er-Li inelastic interspecies collisional properties are studied for magnetic fields up to 680 G. In all cases resonant interspecies loss features, indicative of Feshbach resonances, have been observed. While most resonances have sub-Gauss widths, a few of them are broad and feature widths of several Gauss. Those broad resonances are a key to the realization of ultracold Er-Li quantum gas mixtures with tunable interactions.

cond-mat.quant-gas↗

arXiv2021

Sparse Cholesky factorization by Kullback-Leibler minimization

We propose to compute a sparse approximate inverse Cholesky factor $L$ of a dense covariance matrix $Θ$ by minimizing the Kullback-Leibler divergence between the Gaussian distributions $\mathcal{N}(0, Θ)$ and $\mathcal{N}(0, L^{-\top} L^{-1})$, subject to a sparsity constraint. Surprisingly, this problem has a closed-form solution that can be computed efficiently, recovering the popular Vecchia approximation in spatial statistics. Based on recent results on the approximate sparsity of inverse Cholesky factors of $Θ$ obtained from pairwise evaluation of Green's functions of elliptic boundary-value problems at points $\{x_{i}\}_{1 \leq i \leq N} \subset \mathbb{R}^{d}$, we propose an elimination ordering and sparsity pattern that allows us to compute $ε$-approximate inverse Cholesky factors of such $Θ$ in computational complexity $\mathcal{O}(N \log(N/ε)^d)$ in space and $\mathcal{O}(N \log(N/ε)^{2d})$ in time. To the best of our knowledge, this is the best asymptotic complexity for this class of problems. Furthermore, our method is embarrassingly parallel, automatically exploits low-dimensional structure in the data, and can perform Gaussian-process regression in linear (in $N$) space complexity. Motivated by the optimality properties of our methods, we propose methods for applying it to the joint covariance of training and prediction points in Gaussian-process regression, greatly improving stability and computational cost. Finally, we show how to apply our method to the important setting of Gaussian processes with additive noise, sacrificing neither accuracy nor computational complexity.

math.NA↗

arXiv2020

Compression, inversion, and approximate PCA of dense kernel matrices at near-linear computational complexity

Dense kernel matrices $Θ\in \mathbb{R}^{N \times N}$ obtained from point evaluations of a covariance function $G$ at locations $\{ x_{i} \}_{1 \leq i \leq N} \subset \mathbb{R}^{d}$ arise in statistics, machine learning, and numerical analysis. For covariance functions that are Green's functions of elliptic boundary value problems and homogeneously-distributed sampling points, we show how to identify a subset $S \subset \{ 1 , \dots , N \}^2$, with $\# S = O ( N \log (N) \log^{d} ( N /ε) )$, such that the zero fill-in incomplete Cholesky factorisation of the sparse matrix $Θ_{ij} 1_{( i, j ) \in S}$ is an $ε$-approximation of $Θ$. This factorisation can provably be obtained in complexity $O ( N \log( N ) \log^{d}( N /ε) )$ in space and $O ( N \log^{2}( N ) \log^{2d}( N /ε) )$ in time, improving upon the state of the art for general elliptic operators; we further present numerical evidence that $d$ can be taken to be the intrinsic dimension of the data set rather than that of the ambient space. The algorithm only needs to know the spatial configuration of the $x_{i}$ and does not require an analytic representation of $G$. Furthermore, this factorization straightforwardly provides an approximate sparse PCA with optimal rate of convergence in the operator norm. Hence, by using only subsampling and the incomplete Cholesky factorization, we obtain, at nearly linear complexity, the compression, inversion and approximate PCA of a large class of covariance matrices. By inverting the order of the Cholesky factorization we also obtain a solver for elliptic PDE with complexity $O ( N \log^{d}( N /ε) )$ in space and $O ( N \log^{2d}( N /ε) )$ in time, improving upon the state of the art for general elliptic operators.

math.NA↗

arXiv2020

Implicit competitive regularization in GANs

To improve the stability of GAN training we need to understand why they can produce realistic samples. Presently, this is attributed to properties of the divergence obtained under an optimal discriminator. This argument has a fundamental flaw: If we do not impose regularity of the discriminator, it can exploit visually imperceptible errors of the generator to always achieve the maximal generator loss. In practice, gradient penalties are used to regularize the discriminator. However, this needs a metric on the space of images that captures visual similarity. Such a metric is not known, which explains the limited success of gradient penalties in stabilizing GANs. We argue that the performance of GANs is instead due to the implicit competitive regularization (ICR) arising from the simultaneous optimization of generator and discriminator. ICR promotes solutions that look real to the discriminator and thus leverages its inductive biases to generate realistic images. We show that opponent-aware modelling of generator and discriminator, as present in competitive gradient descent (CGD), can significantly strengthen ICR and thus stabilize GAN training without explicit regularization. In our experiments, we use an existing implementation of WGAN-GP and show that by training it with CGD we can improve the inception score (IS) on CIFAR10 for a wide range of scenarios, without any hyperparameter tuning. The highest IS is obtained by combining CGD with the WGAN-loss, without any explicit regularization.

cs.LG↗

arXiv2020

Competitive Gradient Descent

We introduce a new algorithm for the numerical computation of Nash equilibria of competitive two-player games. Our method is a natural generalization of gradient descent to the two-player setting where the update is given by the Nash equilibrium of a regularized bilinear local approximation of the underlying game. It avoids oscillatory and divergent behaviors seen in alternating gradient descent. Using numerical experiments and rigorous analysis, we provide a detailed comparison to methods based on \emph{optimism} and \emph{consensus} and show that our method avoids making any unnecessary changes to the gradient dynamics while achieving exponential (local) convergence for (locally) convex-concave zero sum games. Convergence and stability properties of our method are robust to strong interactions between the players, without adapting the stepsize, which is not the case with previous methods. In our numerical experiments on non-convex-concave problems, existing methods are prone to divergence and instability due to their sensitivity to interactions among the players, whereas we never observe divergence of our algorithm. The ability to choose larger stepsizes furthermore allows our algorithm to achieve faster convergence, as measured by the number of model evaluations.

math.OC↗

arXiv2020

Competitive Mirror Descent

Constrained competitive optimization involves multiple agents trying to minimize conflicting objectives, subject to constraints. This is a highly expressive modeling language that subsumes most of modern machine learning. In this work we propose competitive mirror descent (CMD): a general method for solving such problems based on first order information that can be obtained by automatic differentiation. First, by adding Lagrange multipliers, we obtain a simplified constraint set with an associated Bregman potential. At each iteration, we then solve for the Nash equilibrium of a regularized bilinear approximation of the full problem to obtain a direction of movement of the agents. Finally, we obtain the next iterate by following this direction according to the dual geometry induced by the Bregman potential. By using the dual geometry we obtain feasible iterates despite only solving a linear system at each iteration, eliminating the need for projection steps while still accounting for the global nonlinear structure of the constraint set. As a special case we obtain a novel competitive multiplicative weights algorithm for problems on the positive cone.

math.OC↗

arXiv2020

Tools for quantum simulation with ultracold atoms in optical lattices

After many years of development of the basic tools, quantum simulation with ultracold atoms has now reached the level of maturity where it can be used to investigate complex quantum processes. Planning of new experiments and upgrading existing set-ups depends crucially on a broad overview of the available techniques, their specific advantages and limitations. This Technical Review aims to provide a comprehensive compendium of the state of the art. We discuss the basic principles, the available techniques and their current range of applications. Focusing on the simulation of varied phenomena in solid-state physics using optical lattice experiments, we review their basics, the necessary techniques and the accessible physical parameters. We outline how to control and use interactions with external potentials and between the atoms, and how to design new synthetic gauge fields and spin-orbit coupling. We discuss the latest progress in site-resolved techniques using quantum gas microscopes, and describe the unique features of quantum simulation experiments with two-electron atomic species.

cond-mat.quant-gas↗

arXiv2020

Comparison of Meta-Heuristics for the Planning of Meshed Power Systems

The power system planning task is a combinatorial optimization problem. The objective function minimizes the economic costs subject to a set of technical and operational constraints. Meta-heuristics are often used as optimization strategies to find solutions to this problem by combining switching, line reinforcement or new line measures. Common heuristics are GA, PSO, HC, ILS or newer methods such as GWO or FWA. In this paper, we compare these algorithms within the same framework. We test each algorithm on 8 different test grids ranging from 73 to 9421 buses. For each grid and algorithm, we start 50 runs with a maximum run time of 1 hour. The results show that the performance of an algorithm depends on the initial grid state, grid size and amount of measures. The ILS method is very robust in most cases. In the larger test grids, more exploratory heuristics, e.g., GA and PSO, find solutions in shorter run times.

eess.SY↗

arXiv2019

Ultracold collisions in the Yb-Li mixture system

We report our experimental results on the collisional physics between non-S-state atoms (ytterbium (Yb), effectively a two-electron system, in the metastable ${}^3\mathrm{P}_2$ state) and S-state atoms (lithium (Li), an alkali metal, in the ground state). At low magnetic fields, by measuring inelastic interspecies collisional losses in the double quantum degenerate mixture we reveal the strong dependence of the inelastic losses on the internal spin states of both species and suppressed losses in stretched state configurations. Increasing the magnetic field up to 800 G we further investigate the magnetic field dependence of the collisional interactions. There, smoothly increasing inelastic losses are observed towards higher fields. The combined knowledge of both the magnetic field and the spin state dependence of the collisional losses of this prototypical mixture system of non-S-state and S-state atoms provides a significant step forward towards controllable impurity physics realized in the Yb-Li ultracold system.

cond-mat.quant-gas↗

arXiv2018

Experimental realization of ultracold Yb-$^{7}{\rm Li}$ mixtures in mixed dimensions

We report on the experimental realization of ultracold $^{174}{\rm Yb}$-$^{7}{\rm Li}$ (Boson-Boson) and $^{173}{\rm Yb}$-$^{7}{\rm Li}$ (Fermion-Boson) mixtures. They are loaded into three dimensional (3D) or one dimensional (1D) optical lattices that are species-selectively deep for the heavy Ytterbium (Yb) and shallow for the light bosonic Lithium (Li) component, realizing novel mixed dimensional systems. In the 1D optical lattice the band structure of $^{173}{\rm Yb}$ is reconstructed in the presence of $^{7}{\rm Li}$. Spectroscopic measurements of the $^{174}{\rm Yb}$-$^{7}{\rm Li}$ mixture in the 3D lattice give access to the $^{174}{\rm Yb}$ Mott-insulator structure. Ground state inter-species scattering lengths are determined to be $|a_{\rm bg}(^{174}{\rm Yb}$-$^{7}{\rm Li})|=(1.11 \pm 0.17)~{\rm nm}$ and $|a_{\rm bg}(^{173}{\rm Yb}$-$^{7}{\rm Li})|=(1.16 \pm 0.18)~{\rm nm}$. The formation and characterization of an ultracold $^{173}{\rm Yb}$-$^{7}{\rm Li}$ mixture is a first step towards a possible realization of a topological $p_x + i\,p_y$ superfluid in this system.

cond-mat.quant-gas↗