Mirco Milletarí | Searcharxiv

arXiv2019

Mean Field Theory of Activation Functions in Deep Neural Networks

We present a Statistical Mechanics (SM) model of deep neural networks, connecting the energy-based and the feed forward networks (FFN) approach. We infer that FFN can be understood as performing three basic steps: encoding, representation validation and propagation. From the meanfield solution of the model, we obtain a set of natural activations -- such as Sigmoid, $\tanh$ and ReLu -- together with the state-of-the-art, Swish; this represents the expected information propagating through the network and tends to ReLu in the limit of zero noise.We study the spectrum of the Hessian on an associated classification task, showing that Swish allows for more consistent performances over a wider range of network architectures.

cs.LG↗

arXiv2017

Optimal charge-to-spin conversion in graphene on transition metal dichalcogenides

When graphene is placed on a monolayer of semiconducting transition metal dichalcogenide (TMD) its band structure develops rich spin textures due to proximity spin-orbital effects with interfacial breaking of inversion symmetry. In this work, we show that the characteristic spin winding of low-energy states in graphene on TMD monolayer enables current-driven spin polarization known as the inverse spin galvanic effect (ISGE). By introducing a proper figure of merit, we quantify the efficiency of charge-to-spin conversion and show it is close to unity when the Fermi level approaches the spin minority band. Remarkably, at high electronic density, even though sub-bands with opposite spin helicities are occupied, the efficiency decays only algebraically. The giant ISGE predicted for graphene on TMD monolayer is robust against disorder and remains large at room temperature.

cond-mat.mes-hall↗