arXiv subjects

Chase Goddard

Publications and source records attributed to Chase Goddard.

4 recordsLinked to original sources

arXiv2025

Entropic Confinement and Mode Connectivity in Overparameterized Neural Networks

Modern neural networks exhibit a striking property: basins of attraction in the loss landscape are often connected by low-loss paths, yet optimization dynamics generally remain confined to a single convex basin and rarely explore intermediate points. We resolve this paradox by identifying entropic barriers arising from the interplay between curvature variations along these paths and noise in optimization dynamics. Empirically, we find that curvature systematically rises away from minima, producing effective forces that bias noisy dynamics back toward the endpoints - even when the loss remains nearly flat. These barriers persist longer than energetic barriers, shaping the late-time localization of solutions in parameter space. Our results highlight the role of curvature-induced entropic forces in governing both connectivity and confinement in deep learning landscapes.

cs.LG↗

arXiv2025

When can in-context learning generalize out of task distribution?

In-context learning (ICL) is a remarkable capability of pretrained transformers that allows models to generalize to unseen tasks after seeing only a few examples. We investigate empirically the conditions necessary on the pretraining distribution for ICL to emerge and generalize \emph{out-of-distribution}. Previous work has focused on the number of distinct tasks necessary in the pretraining dataset. Here, we use a different notion of task diversity to study the emergence of ICL in transformers trained on linear functions. We find that as task diversity increases, transformers undergo a transition from a specialized solution, which exhibits ICL only within the pretraining task distribution, to a solution which generalizes out of distribution to the entire task space. We also investigate the nature of the solutions learned by the transformer on both sides of the transition, and observe similar transitions in nonlinear regression problems. We construct a phase diagram to characterize how our concept of task diversity interacts with the number of pretraining tasks. In addition, we explore how factors such as the depth of the model and the dimensionality of the regression problem influence the transition.

cs.LG↗

arXiv2025

Optimization and variability can coexist

Many biological systems perform close to their physical limits, but promoting this optimality to a general principle seems to require implausibly fine tuning of parameters. Using examples from a wide range of systems, we show that this intuition is wrong. Near an optimum, functional performance depends on parameters in a "sloppy'' way, with some combinations of parameters being only weakly constrained. Absent any other constraints, this predicts that we should observe widely varying parameters, and we make this precise: the entropy in parameter space can be extensive even if performance on average is very close to optimal. This removes a major objection to optimization as a general principle, and rationalizes the observed variability.

q-bio.QM↗

arXiv2021

Testing for the Continuous Spectrum of X-Rays Predicted to Accompany the Photoejection of an Atomic Inner Shell Electron

Echoing classical physics, quantum electrodynamics predicts the release of a spectral continuum of electromagnetic radiation upon the sudden acceleration of charged particles in quantum matter. Despite apparent theoretical success in describing sister nuclear processes, known as internal bremsstrahlung, following nuclear beta decay and K capture, the situation of the photoejection of an electron from an inner shell of an atom, intraatomic bremsstrahlung (IAB), is far from settled. In this paper we present fresh measurements which rely on contemporary signal processing as well as the high flux available from a synchrotron radiation source to revisit the problem by photoejecting electrons from the innermost shell of copper. For the first time we have sufficient sample statistics to measure the expected spectra at the level expected by contemporary theory. Furthermore, we employ sufficiently thin targets to overcome secondary scattering artifacts. Our approach applies the fluorescence coincidence method to guard against extraneous scattering and multiple incident photon processes. Our observations set a severe upper limit on the rate for IAB: We conclude that current theory overpredicts, by at least 5 sigma, the measured rate for K shell IAB in copper in the range of detected energies below the K fluorescence energy.

physics.atom-ph↗