arXiv subjects

Benoit Dherin

Publications and source records attributed to Benoit Dherin.

36 records · Page 2Linked to original sources

arXiv2021

The Geometric Occam's Razor Implicit in Deep Learning

In over-parameterized deep neural networks there can be many possible parameter configurations that fit the training data exactly. However, the properties of these interpolating solutions are poorly understood. We argue that over-parameterized neural networks trained with stochastic gradient descent are subject to a Geometric Occam's Razor; that is, these networks are implicitly regularized by the geometric model complexity. For one-dimensional regression, the geometric model complexity is simply given by the arc length of the function. For higher-dimensional settings, the geometric model complexity depends on the Dirichlet energy of the function. We explore the relationship between this Geometric Occam's Razor, the Dirichlet energy and other known forms of implicit regularization. Finally, for ResNets trained on CIFAR-10, we observe that Dirichlet energy measurements are consistent with the action of this implicit Geometric Occam's Razor.

cs.LG↗

arXiv2021

Discretization Drift in Two-Player Games

Gradient-based methods for two-player games produce rich dynamics that can solve challenging problems, yet can be difficult to stabilize and understand. Part of this complexity originates from the discrete update steps given by simultaneous or alternating gradient descent, which causes each player to drift away from the continuous gradient flow -- a phenomenon we call discretization drift. Using backward error analysis, we derive modified continuous dynamical systems that closely follow the discrete dynamics. These modified dynamics provide an insight into the notorious challenges associated with zero-sum games, including Generative Adversarial Networks. In particular, we identify distinct components of the discretization drift that can alter performance and in some cases destabilize the game. Finally, quantifying discretization drift allows us to identify regularizers that explicitly cancel harmful forms of drift or strengthen beneficial forms of drift, and thus improve performance of GAN training.

stat.ML↗

arXiv2021

On the Origin of Implicit Regularization in Stochastic Gradient Descent

For infinitesimal learning rates, stochastic gradient descent (SGD) follows the path of gradient flow on the full batch loss function. However moderately large learning rates can achieve higher test accuracies, and this generalization benefit is not explained by convergence bounds, since the learning rate which maximizes test accuracy is often larger than the learning rate which minimizes training loss. To interpret this phenomenon we prove that for SGD with random shuffling, the mean SGD iterate also stays close to the path of gradient flow if the learning rate is small and finite, but on a modified loss. This modified loss is composed of the original loss function and an implicit regularizer, which penalizes the norms of the minibatch gradients. Under mild assumptions, when the batch size is small the scale of the implicit regularization term is proportional to the ratio of the learning rate to the batch size. We verify empirically that explicitly including the implicit regularizer in the loss can enhance the test accuracy when the learning rate is small.

cs.LG↗

arXiv2020

Symplectic Microgeometry IV: Quantization

We construct a special class of semiclassical Fourier integral operators whose wave fronts are symplectic micromorphisms. These operators have very good properties: they form a category on which the wave front map becomes a functor into the cotangent microbundle category, and they admit a total symbol calculus in terms of symplectic micromorphisms enhanced with half-density germs. This new operator category encompasses the semi-classical pseudo-differential calculus and offers a functorial framework for the semi-classical analysis of the Schrödinger equation. We also comment on applications to classical and quantum mechanics as well as to a functorial and geometrical approach to the quantization of Poisson manifolds.

math.SG↗

arXiv2015

Formal symplectic realizations

We study the relationship between several constructions of symplectic realizations of a given Poisson manifold. Our main result is a general formula for a formal symplectic realization in the case of an arbitrary Poisson structure on $\R^n$. This formula is expressed in terms of rooted trees and elementary differentials, building on the work of Butcher, and the coefficients are shown to be a generalization of Bernoulli numbers appearing in the linear Poisson case. We also show that this realization coincides with a formal version of the original construction of Weinstein, when suitably put in global Darboux form, and with the realization coming from tree-level part of Kontsevich's star product. We provide a simple iterated integral expression for the relevant coefficients and show that they coincide with underlying Kontsevich weights.

math.SG↗

arXiv2014

Deformation quantization of Leibniz algebras

This paper has two parts. The first part is a review and extension of the methods of integration of Leibniz algebras into Lie racks, including as new feature a new way of integrating 2-cocycles (see Lemma 3.9). In the second part, we use the local integration of a Leibniz algebra h using a Baker-Campbell-Hausdorff type formula in order to deformation quantize its linear dual h^*. More precisely, we define a natural rack product on the set of exponential functions which extends to a rack action on C^{\infty}(h^*).

math.SG↗

arXiv2014

G-Systems and Deformation of G-Actions On Rd

Given a (smooth) action of a Lie group G on Rd we construct a DGA whose Maurer-Cartan elements are in one to one correspondence with some class of defomations of the (induced) G-action on the ring of formal power series with coefficients in the ring of smooth functions on Rd. In the final part of this note we discuss the cohomological obstructions to the existence and to the uniqueness (in a sense to be clarified) of such deformations.

math-ph↗

arXiv2012

Quantizations of Momentum Maps and G-Systems

In this note, we give an explicit formula for a family of deformation quantizations for the momentum map associated with the cotangent lift of a Lie group action on Rd. This family of quantizations is parametrized by the formal G-systems introduced by the authors in a previous paper and allows us to obtain classical invariant Hamiltonians that quantize without anomalies with respect to the quantizations of the action prescribed by the formal G-systems.

math-ph↗

arXiv2012

Integration of Lie Algebroid Comorphisms

We show that the path construction integration of Lie algebroids by Lie groupoids is an actual equivalence from the category of integrable Lie algebroids and complete Lie algebroid comorphisms to the category of source 1-connected Lie groupoids and Lie groupoid comorphisms. This allows us to construct an actual symplectization functor in Poisson geometry. We include examples to show that the integrability of comorphisms and Poisson maps may not hold in the absence of a completeness assumption.

math.DG↗

arXiv2012

Quantization of (volume-preserving) actions on R^d

We associate a space of (formal) representations on the space of h-formal power series with coefficients in the space of smooth functions on Rd (which we call quantizations) with an action of a group on Rd by smooth diffeomorphisms. If the action is further volume preserving, these quantizations can be realized as unitary representations on square summable functions on Rd by bounded h-dependent Fourier integral operators, the formal case corresponding to the asymptotics in the limit h going to zero. We construct DGAs controlling these quantizations and prove existence and rigidity results for them.

math-ph↗

arXiv2011

Symplectic Microgeometry III: Monoids

We show that the category of Poisson manifolds and Poisson maps, the category of symplectic microgroupoids and lagrangian submicrogroupoids (as morphisms), and the category of monoids and monoid morphisms in the microsymplectic category are equivalent symmetric monoidal categories.

math.SG↗

arXiv2011

Symplectic Microgeometry II: Generating functions

We adapt the notion of generating functions for lagrangian submanifolds to symplectic microgeometry. We show that a symplectic micromorphism always admits a global generating function. As an application, we describe hamiltonian flows as special symplectic micromorphisms whose local generating functions are the solutions of Hamilton-Jacobi equations. We obtain a purely categorical formulation of the temporal evolution in classical mechanics.

math.SG↗

arXiv2010

Tensor products of representations up to homotopy

We study the construction of tensor products of representations up to homotopy, which are the A-infinity version of ordinary representations. We provide formulas for the construction of tensor products of representations up to homotopy and of morphisms between them, and show that these formulas give the homotopy category a monoidal structure which is uniquely defined up to equivalence.

math.AT↗

arXiv2009

Symplectic Microgeometry I: Micromorphisms

We introduce the notion of symplectic microfolds and symplectic micromorphisms between them. They form a monoidal category, which is a version of the "category" of symplectic manifolds and canonical relations obtained by localizing them around lagrangian submanifolds in the spirit of Milnor's microbundles.

math.SG↗

arXiv2007

Cotangent Microbundle Category, I

We define a local version of the extended symplectic category, the cotangent microbundle category, MiC, which turns out to be a true monoidal category. We show that a monoid in this category induces a Poisson manifold together with the local symplectic groupoid integrating it. Moreover, we prove that monoid morphisms produce Poisson maps between the induced Poisson manifolds in a functorial way. This gives a functor between the category of monoids in MiC and the category of Poisson manifolds and Poisson maps. Conversely, the semi-classical part of the Kontsevich star-product associated to a real-analytic Poisson structure on an open subset of R^n produces a monoid in MiC.

math-ph↗

arXiv2005

The Universal Generating Function of Analytical Poisson Structures

The notion of generating functions of Poisson structures was first studied in math.SG/0312380.They are special functions which induce, on open subsets of $\R^d$, a Poisson structure together with the local symplectic groupoid integrating it. A universal generating function was provided in terms of a formal power series coming from Kontsevich star product. The present article proves that this universal generating function converges for analytical Poisson structures and compares the induced local symplectic groupoid with the phase space of Karasev--Maslov.

math.SG↗

arXiv2005

Formal Lagrangian Operad

Given a symplectic manifold $M$, we may define an operad structure on the the spaces $\op^k$ of the Lagrangian submanifolds of $(\bar{M})^k\times M$ via symplectic reduction. If $M$ is also a symplectic groupoid, then its multiplication space is an associative product in this operad. Following this idea, we provide a deformation theory for symplectic groupoids analog to the deformation theory of algebras. It turns out that the semi-classical part of Kontsevich's deformation of $C^\infty(\R^d)$ is a deformation of the trivial symplectic groupoid structure of $T^*\R^d$.

math.SG↗

arXiv2003

Formal symplectic groupoid

The multiplicative structure of the trivial symplectic groupoid over $\mathbb R^d$ associated to the zero Poisson structure can be expressed in terms of a generating function. We address the problem of deforming such a generating function in the direction of a non-trivial Poisson structure so that the multiplication remains associative. We prove that such a deformation is unique under some reasonable conditions and we give the explicit formula for it. This formula turns out to be the semi-classical approximation of Kontsevich's deformation formula. For the case of a linear Poisson structure, the deformed generating function reduces exactly to the CBH formula of the associated Lie algebra. The methods used to prove existence are interesting in their own right as they come from an at first sight unrelated domain of mathematics: the Runge--Kutta theory of the numeric integration of ODE's.

math.SG↗