arXiv subjects

James H. von Brecht

Publications and source records attributed to James H. von Brecht.

7 recordsLinked to original sources

arXiv2023

Fronts under arrest II: analytical foundations

We study a class of minimal geometric partial differential equations that serves as a framework to understand the evolution of boundaries between states in different pattern forming systems. The framework combines normal growth, curvature flow and nonlocal interaction terms to track the motion of these interfaces. This approach was first developed to understand arrested fronts in a bacterial system. These are fronts that become stationary as they grow into each other. This paper establishes analytic foundations and geometric insight for studying this class of equations. In so doing, an efficient numerical scheme is developed and employed to gain further insight into the dynamics of these general pattern forming systems.

math.AP↗

arXiv2023

Feature Collapse

We formalize and study a phenomenon called feature collapse that makes precise the intuitive idea that entities playing a similar role in a learning task receive similar representations. As feature collapse requires a notion of task, we leverage a simple but prototypical NLP task to study it. We start by showing experimentally that feature collapse goes hand in hand with generalization. We then prove that, in the large sample limit, distinct words that play identical roles in this NLP task receive identical local feature representations in a neural network. This analysis reveals the crucial role that normalization mechanisms, such as LayerNorm, play in feature collapse and in generalization.

cs.LG↗

arXiv2022

Long-Tailed Learning Requires Feature Learning

We propose a simple data model inspired from natural data such as text or images, and use it to study the importance of learning features in order to achieve good generalization. Our data model follows a long-tailed distribution in the sense that some rare subcategories have few representatives in the training set. In this context we provide evidence that a learner succeeds if and only if it identifies the correct features, and moreover derive non-asymptotic generalization error bounds that precisely quantify the penalty that one must pay for not learning features.

cs.LG↗

arXiv2017

Dynamics of Embedded Curves by Doubly-Nonlocal Reaction-Diffusion Systems

We study a class of nonlocal, energy-driven dynamical models that govern the motion of closed, embedded curves from both an energetic and dynamical perspective. Our energetic results provide a variety of ways to understand physically motivated energetic models in terms of more classical, combinatorial measures of complexity for embedded curves. This line of investigation culminates in a family of complexity bounds that relate a rather broad class of models to a generalized, or weighted, variant of the crossing number. Our dynamic results include global well-posedness of the associated partial differential equations, regularity of equilibria for these flows as well as a more detailed investigation of dynamics near such equilibria. Finally, we explore a few global dynamical properties of these models numerically.

math.AP↗

arXiv2013

Multiclass Total Variation Clustering

Ideas from the image processing literature have recently motivated a new set of clustering algorithms that rely on the concept of total variation. While these algorithms perform well for bi-partitioning tasks, their recursive extensions yield unimpressive results for multiclass clustering tasks. This paper presents a general framework for multiclass total variation clustering that does not rely on recursion. The results greatly outperform previous total variation algorithms and compare well with state-of-the-art NMF approaches.

stat.ML↗

arXiv2013

An Adaptive Total Variation Algorithm for Computing the Balanced Cut of a Graph

We propose an adaptive version of the total variation algorithm proposed in [3] for computing the balanced cut of a graph. The algorithm from [3] used a sequence of inner total variation minimizations to guarantee descent of the balanced cut energy as well as convergence of the algorithm. In practice the total variation minimization step is never solved exactly. Instead, an accuracy parameter is specified and the total variation minimization terminates once this level of accuracy is reached. The choice of this parameter can vastly impact both the computational time of the overall algorithm as well as the accuracy of the result. Moreover, since the total variation minimization step is not solved exactly, the algorithm is not guarantied to be monotonic. In the present work we introduce a new adaptive stopping condition for the total variation minimization that guarantees monotonicity. This results in an algorithm that is actually monotonic in practice and is also significantly faster than previous, non-adaptive algorithms.

math.OC↗

arXiv2012

Convergence of a Steepest Descent Algorithm for Ratio Cut Clustering

Unsupervised clustering of scattered, noisy and high-dimensional data points is an important and difficult problem. Tight continuous relaxations of balanced cut problems have recently been shown to provide excellent clustering results. In this paper, we present an explicit-implicit gradient flow scheme for the relaxed ratio cut problem, and prove that the algorithm converges to a critical point of the energy. We also show the efficiency of the proposed algorithm on the two moons dataset.

math.OC↗