arXiv subjects

Noam Levi

Publications and source records attributed to Noam Levi.

28 records · Page 2Linked to original sources

arXiv2023

The Universal Statistical Structure and Scaling Laws of Chaos and Turbulence

Turbulence is a complex spatial and temporal structure created by the strong non-linear dynamics of fluid flows at high Reynolds numbers. Despite being an ubiquitous phenomenon that has been studied for centuries, a full understanding of turbulence remained a formidable challenge. Here, we introduce tools from the fields of quantum chaos and Random Matrix Theory (RMT) and present a detailed analysis of image datasets generated from turbulence simulations of incompressible and compressible fluid flows. Focusing on two observables: the data Gram matrix and the single image distribution, we study both the local and global eigenvalue statistics and compare them to classical chaos, uncorrelated noise and natural images. We show that from the RMT perspective, the turbulence Gram matrices lie in the same universality class as quantum chaotic rather than integrable systems, and the data exhibits power-law scalings in the bulk of its eigenvalues which are vastly different from uncorrelated classical chaos, random data, natural images. Interestingly, we find that the single sample distribution only appears as fully RMT chaotic, but deviates from chaos at larger correlation lengths, as well as exhibiting different scaling properties.

cond-mat.stat-mech↗

arXiv2023

Grokking in Linear Estimators -- A Solvable Model that Groks without Understanding

Grokking is the intriguing phenomenon where a model learns to generalize long after it has fit the training data. We show both analytically and numerically that grokking can surprisingly occur in linear networks performing linear tasks in a simple teacher-student setup with Gaussian inputs. In this setting, the full training dynamics is derived in terms of the training and generalization data covariance matrix. We present exact predictions on how the grokking time depends on input and output dimensionality, train sample size, regularization, and network initialization. We demonstrate that the sharp increase in generalization accuracy may not imply a transition from "memorization" to "understanding", but can simply be an artifact of the accuracy measure. We provide empirical verification for our calculations, along with preliminary results indicating that some predictions also hold for deeper networks, with non-linear activations.

stat.ML↗

arXiv2023

Charting the Topography of the Neural Network Landscape with Thermal-Like Noise

The training of neural networks is a complex, high-dimensional, non-convex and noisy optimization problem whose theoretical understanding is interesting both from an applicative perspective and for fundamental reasons. A core challenge is to understand the geometry and topography of the landscape that guides the optimization. In this work, we employ standard Statistical Mechanics methods, namely, phase-space exploration using Langevin dynamics, to study this landscape for an over-parameterized fully connected network performing a classification task on random data. Analyzing the fluctuation statistics, in analogy to thermal dynamics at a constant temperature, we infer a clear geometric description of the low-loss region. We find that it is a low-dimensional manifold whose dimension can be readily obtained from the fluctuations. Furthermore, this dimension is controlled by the number of data points that reside near the classification decision boundary. Importantly, we find that a quadratic approximation of the loss near the minimum is fundamentally inadequate due to the exponential nature of the decision boundary and the flatness of the low-loss region. This causes the dynamics to sample regions with higher curvature at higher temperatures, while producing quadratic-like statistics at any given temperature. We explain this behavior by a simplified loss model which is analytically tractable and reproduces the observed fluctuation statistics.

cond-mat.stat-mech↗

arXiv2022

Detection of Early-Universe Gravitational Wave Signatures and Fundamental Physics

Detection of a gravitational-wave signal of non-astrophysical origin would be a landmark discovery, potentially providing a significant clue to some of our most basic, big-picture scientific questions about the Universe. In this white paper, we survey the leading early-Universe mechanisms that may produce a detectable signal -- including inflation, phase transitions, topological defects, as well as primordial black holes -- and highlight the connections to fundamental physics. We review the complementarity with collider searches for new physics, and multimessenger probes of the large-scale structure of the Universe.

gr-qc↗

arXiv2022

Noise Injection Node Regularization for Robust Learning

We introduce Noise Injection Node Regularization (NINR), a method of injecting structured noise into Deep Neural Networks (DNN) during the training stage, resulting in an emergent regularizing effect. We present theoretical and empirical evidence for substantial improvement in robustness against various test data perturbations for feed-forward DNNs when trained under NINR. The novelty in our approach comes from the interplay of adaptive noise injection and initialization conditions such that noise is the dominant driver of dynamics at the start of training. As it simply requires the addition of external nodes without altering the existing network structure or optimization algorithms, this method can be easily incorporated into many standard problem specifications. We find improved stability against a number of data perturbations, including domain shifts, with the most dramatic improvement obtained for unstructured noise, where our technique outperforms other existing methods such as Dropout or $L_2$ regularization, in some cases. We further show that desirable generalization properties on clean data are generally maintained.

cs.LG↗

arXiv2022

Noise Injection as a Probe of Deep Learning Dynamics

We propose a new method to probe the learning mechanism of Deep Neural Networks (DNN) by perturbing the system using Noise Injection Nodes (NINs). These nodes inject uncorrelated noise via additional optimizable weights to existing feed-forward network architectures, without changing the optimization algorithm. We find that the system displays distinct phases during training, dictated by the scale of injected noise. We first derive expressions for the dynamics of the network and utilize a simple linear model as a test case. We find that in some cases, the evolution of the noise nodes is similar to that of the unperturbed loss, thus indicating the possibility of using NINs to learn more about the full system in the future.

cs.LG↗

arXiv2022

Report of the Topical Group on Cosmic Probes of Fundamental Physics for for Snowmass 2021

Cosmic Probes of Fundamental Physics take two primary forms: Very high energy particles (cosmic rays, neutrinos, and gamma rays) and gravitational waves. Already today, these probes give access to fundamental physics not available by any other means, helping elucidate the underlying theory that completes the Standard Model. The last decade has witnessed a revolution of exciting discoveries such as the detection of high-energy neutrinos and gravitational waves. The scope for major developments in the next decades is dramatic, as we detail in this report.

hep-ph↗

arXiv2020

Ripples in Spacetime from Broken Supersymmetry

We initiate the study of gravitational wave (GW) signals from first-order phase transitions in supersymmetry-breaking hidden sectors. Such phase transitions often occur along a pseudo-flat direction universally related to supersymmetry (SUSY) breaking in hidden sectors that spontaneously break $R$-symmetry. The potential along this pseudo-flat direction imbues the phase transition with a number of novel properties, including a nucleation temperature well below the scale of heavy states (such that the temperature dependence is captured by the low-temperature expansion) and significant friction induced by the same heavy states as they pass through bubble walls. In low-energy SUSY-breaking hidden sectors, the frequency of the GW signal arising from such a phase transition is guaranteed to lie within the reach of future interferometers given existing cosmological constraints on the gravitino abundance. Once a mediation scheme is specified, the frequency of the GW peak correlates with the superpartner spectrum. Current bounds on supersymmetry are compatible with GW signals at future interferometers, while the observation of a GW signal from a SUSY-breaking hidden sector would imply superpartners within reach of future colliders.

hep-ph↗

arXiv2020

Heavy Thermal Relics from Zombie Collisions

We propose a new thermal freezeout mechanism which results in dark matter masses exceeding the unitarity bound by many orders of magnitude, without violating perturbative unitarity or modifying the standard cosmology. The process determining the relic abundance is $χζ^\dagger \to ζζ$, where $χ$ is the dark matter candidate. For $ m_ζ< m_χ< 3 m_ζ$, $χ$ is cosmologically long-lived and scatters against the exponentially more abundant $ζ$. Therefore, such a process allows for exponentially heavier dark matter for the same interaction strength as a particle undergoing ordinary $2 \to 2$ freezeout; or equivalently, exponentially weaker interactions for the same mass. We demonstrate this mechanism in a leptophilic dark matter model, which allows for dark matter masses up to $10^{9}$ GeV.

hep-ph↗

arXiv2018

Light Dark Matter from Leptogenesis

We consider the implications of a shared production mechanism between the baryon asymmetry of the universe and the relic abundance of dark matter, that does not result in matching asymmetries. We present a simple model within a two sector leptogenesis framework, in which right handed sterile neutrinos decay out of equilibrium to both the Standard Model and the dark sector, generating an asymmetry in one and populating the other. This realization naturally accommodates light dark matter in the keV mass scale and above. Interactions in the dark sector may or may not cause the sector to thermalize, leading to interesting phenomenological implications, including hot, warm or cold thermal relic dark matter, while evading cosmological constraints. Under minimal assumptions the model provides a novel non-thermal production mechanism for sterile neutrino dark matter and predicts indirect detection signatures which may address the unexplained 3.5 keV line observed in various galaxy clusters.

hep-ph↗