arXiv subjects

Y. Cooper

Publications and source records attributed to Y. Cooper.

5 recordsLinked to original sources

arXiv2023

How regularization affects the geometry of loss functions

What neural networks learn depends fundamentally on the geometry of the underlying loss function. We study how different regularizers affect the geometry of this function. One of the most basic geometric properties of a smooth function is whether it is Morse or not. For nonlinear deep neural networks, the unregularized loss function $L$ is typically not Morse. We consider several different regularizers, including weight decay, and study for which regularizers the regularized function $L_\epsilon$ becomes Morse.

cs.LG↗

arXiv2020

The critical locus of overparameterized neural networks

Many aspects of the geometry of loss functions in deep learning remain mysterious. In this paper, we work toward a better understanding of the geometry of the loss function $L$ of overparameterized feedforward neural networks. In this setting, we identify several components of the critical locus of $L$ and study their geometric properties. For networks of depth $\ell \geq 4$, we identify a locus of critical points we call the star locus $S$. Within $S$ we identify a positive-dimensional sublocus $C$ with the property that for $p \in C$, $p$ is a degenerate critical point, and no existing theoretical result guarantees that gradient descent will not converge to $p$. For very wide networks, we build on earlier work and show that all critical points of $L$ are degenerate, and give lower bounds on the number of zero eigenvalues of the Hessian at each critical point. For networks that are both deep and very wide, we compare the growth rates of the zero eigenspaces of the Hessian at all the different families of critical points that we identify. The results in this paper provide a starting point to a more quantitative understanding of the properties of various components of the critical locus of $L$.

cs.LG↗

arXiv2020

A comparison of group testing architectures for COVID-19 testing

An important component of every country's COVID-19 response is fast and efficient testing - to identify and isolate cases, as well as for early detection of local hotspots. For many countries, producing a sufficient number of tests has been a serious limiting factor in their efforts to control COVID-19 infections. Group testing is a well-established mathematical tool, which can provide a substantial and inexpensive expansion of testing capacity. In this note, we compare several popular group testing schemes in the context of qPCR testing for COVID-19. We find that in practical settings, for identification of individuals with COVID-19, Dorfman testing is the best choice at prevalences up to 30%, while for estimation of COVID-19 prevalence rates in the total population, Gibbs-Gower testing is the best choice at prevalences up to 30% given a fixed and relatively small number of tests. For instance, at a prevalence of up to 2%, Dorfman testing gives an efficiency gain of 3.5--8; at 1% prevalence, Gibbs-Gower testing gives an efficiency gain of 18, even when capping the pool size at a feasible number . This note is intended as a helpful handbook for labs implementing group testing methods.

stat.ME↗

arXiv2018

Gradient descent in higher codimension

We consider the behavior of gradient flow and of discrete and noisy gradient descent. It is commonly noted that the addition of noise to the process of discrete gradient descent can affect the trajectory of gradient descent. In previous work, we observed such effects. There, we considered the case where the minima had codimension 1. In this note, we do some computer experiments and observe the behavior of noisy gradient descent in the more complex setting of minima of higher codimension.

math.OC↗

arXiv2018

Gradient descent in some simple settings

In this note, we observe the behavior of gradient flow and discrete and noisy gradient descent in some simple settings. It is commonly noted that addition of noise to gradient descent can affect the trajectory of gradient descent. Here, we run some computer experiments for gradient descent on some simple functions, and observe this principle in some concrete examples.

math.OC↗