arXiv subjects

Stefan Steinerberger

Publications and source records attributed to Stefan Steinerberger.

At least 109 records · Page 6Linked to original sources

arXiv2021

A Nonlocal Transport Equation Describing Roots of Polynomials Under Differentiation

Let $p_n$ be a polynomial of degree $n$ having all its roots on the real line distributed according to a smooth function $u(0,x)$. One could wonder how the distribution of roots behaves under iterated differentation of the function, i.e. how the density of roots of $p_n^{(k)}$ evolves. We derive a nonlinear transport equation with nonlocal flux $$ u_t + \frac{1}π\left( \arctan{ \left( \frac{Hu}{ u}\right)} \right)_x = 0,$$ where $H$ is the Hilbert transform. This equation has three very different compactly supported solutions: (1) the arcsine distribution $u(t,x) = (1-x^2)^{-1/2} χ_{(-1,1)}$, (2) the family of semicircle distributions $$ u(t,x) = \frac{2}π \sqrt{(T-t) - x^2}$$ and (3) a family of solutions contained in the Marchenko-Pastur law.

math.AP↗

arXiv2021

On the Stability of Fourier Phase Retrieval

Phase retrieval is concerned with recovering a function $f$ from the absolute value of its Fourier transform $|\widehat{f}|$. We study the stability properties of this problem in Lebesgue spaces. Our main results shows that $$ \| f-g\|_{L^2(\mathbb{R}^n)} \leq 2\cdot \| |\widehat{f}| - |\widehat{g}| \|_{L^2(\mathbb{R}^n)} + h_f\left( \|f-g\|^{}_{L^p(\mathbb{R}^n)}\right) + J(\widehat{f}, \widehat{g}),$$ where $1 \leq p < 2$, $h_f$ is an explicit nonlinear function depending on the smoothness of $f$ and $J$ is an explicit term capturing the invariance under translations. A noteworthy aspect is that the stability is phrased in terms of $L^p$ for $1 \leq p < 2$ while, usually, $L^p$ cannot be used to control $L^2$, the stability estimate has the flavor of an inverse Hölder inequality. It seems conceivable that the estimate is optimal up to constants.

math.FA↗

arXiv2021

Refined Least Squares for Support Recovery

We study the problem of exact support recovery based on noisy observations and present Refined Least Squares (RLS). Given a set of noisy measurement $$ \myvec{y} = \myvec{X}\myvecθ^* + \myvecω,$$ and $\myvec{X} \in \mathbb{R}^{N \times D}$ which is a (known) Gaussian matrix and $\myvecω \in \mathbb{R}^N$ is an (unknown) Gaussian noise vector, our goal is to recover the support of the (unknown) sparse vector $\myvecθ^* \in \left\{-1,0,1\right\}^D$. To recover the support of the $\myvecθ^*$ we use an average of multiple least squares solutions, each computed based on a subset of the full set of equations. The support is estimated by identifying the most significant coefficients of the average least squares solution. We demonstrate that in a wide variety of settings our method outperforms state-of-the-art support recovery algorithms.

math.ST↗

arXiv2021

Free Convolution Powers via Roots of Polynomials

Let $μ$ be a compactly supported probability measure on the real line. Bercovici-Voiculescu and Nica-Speicher proved the existence of a free convolution power $μ^{\boxplus k}$ for any real $k \geq 1$. The purpose of this short note is to give an elementary description of $μ^{\boxplus k}$ in terms of of polynomials and roots of their derivatives. This bridge allows us to switch back and forth between free probability and the asymptotic behavior of polynomials.

math.PR↗

arXiv2021

t-SNE, Forceful Colorings and Mean Field Limits

t-SNE is one of the most commonly used force-based nonlinear dimensionality reduction methods. This paper has two contributions: the first is forceful colorings, an idea that is also applicable to other force-based methods (UMAP, ForceAtlas2,...). In every equilibrium, the attractive and repulsive forces acting on a particle cancel out: however, both the size and the direction of the attractive (or repulsive) forces acting on a particle are related to its properties: the force vector can serve as an additional feature. Secondly, we analyze the case of t-SNE acting on a single homogeneous cluster (modeled by affinities coming from the adjacency matrix of a random k-regular graph); we derive a mean-field model that leads to interesting questions in classical calculus of variations. The model predicts that, in the limit, the t-SNE embedding of a single perfectly homogeneous cluster is not a point but a thin annulus of diameter $\sim k^{-1/4} n^{-1/4}$. This is supported by numerical results. The mean field ansatz extends to other force-based dimensionality reduction methods.

cs.LG↗

arXiv2021

Max-Cut via Kuramoto-type Oscillators

We consider the Max-Cut problem. Let $G = (V,E)$ be a graph with adjacency matrix $(a_{ij})_{i,j=1}^{n}$. Burer, Monteiro & Zhang proposed to find, for $n$ angles $\left\{θ_1, θ_2, \dots, θ_n\right\} \subset [0, 2π]$, minima of the energy $$ f(θ_1, \dots, θ_n) = \sum_{i,j=1}^{n} a_{ij} \cos{(θ_i - θ_j)}$$ because configurations achieving a global minimum leads to a partition of size 0.878 Max-Cut(G). This approach is known to be computationally viable and leads to very good results in practice. We prove that by replacing $\cos{(θ_i - θ_j)}$ with an explicit function $g_{\varepsilon}(θ_i - θ_j)$ global minima of this new functional lead to a $(1-\varepsilon)$Max-Cut(G). This suggests some interesting algorithms that perform well. It also shows that the problem of finding approximate global minima of energy functionals of this type is NP-hard in general.

math.OC↗

arXiv2021

A Pointwise Inequality for Derivatives of Solutions of the Heat Equation in Bounded Domains

Let $u(t,x)$ be a solution of the heat equation in $\mathbb{R}^n$. Then, each $k-$th derivative also solves the heat equation and satisfies a maximum principle, the largest $k-$th derivative of $u(t,x)$ cannot be larger than the largest $k-$th derivative of $u(0,x)$. We prove an analogous statement for the solution of the heat equation on bounded domains $Ω\subset \mathbb{R}^n$ with Dirichlet boundary conditions. As an application, we give a new and fairly elementary proof of the sharp growth of the second derivatives of Laplacian eigenfunction $-Δϕ_k = λ_k ϕ_k$ with Dirichlet conditions on smooth domains $Ω\subset \mathbb{R}^n$.

math.AP↗

arXiv2021

Neural Collapse with Cross-Entropy Loss

We consider the variational problem of cross-entropy loss with $n$ feature vectors on a unit hypersphere in $\mathbb{R}^d$. We prove that when $d \geq n - 1$, the global minimum is given by the simplex equiangular tight frame, which justifies the neural collapse behavior. We also prove that as $n \rightarrow \infty$ with fixed $d$, the minimizing points will distribute uniformly on the hypersphere and show a connection with the frame potential of Benedetto & Fickus.

cs.LG↗

arXiv2020

The Spectral Underpinning of word2vec

word2vec due to Mikolov \textit{et al.} (2013) is a word embedding method that is widely used in natural language processing. Despite its great success and frequent use, theoretical justification is still lacking. The main contribution of our paper is to propose a rigorous analysis of the highly nonlinear functional of word2vec. Our results suggest that word2vec may be primarily driven by an underlying spectral method. This insight may open the door to obtaining provable guarantees for word2vec. We support these findings by numerical simulations. One fascinating open question is whether the nonlinear properties of word2vec that are not captured by the spectral method are beneficial and, if so, by what mechanism.

cs.LG↗

arXiv2020

Randomly Aggregated Least Squares for Support Recovery

We study the problem of exact support recovery: given an (unknown) vector $θ\in \left\{-1,0,1\right\}^D$, we are given access to the noisy measurement $$ y = Xθ+ ω,$$ where $X \in \mathbb{R}^{N \times D}$ is a (known) Gaussian matrix and the noise $ω\in \mathbb{R}^N$ is an (unknown) Gaussian vector. How small we can choose $N$ and still reliably recover the support of $θ$? We present RAWLS (Randomly Aggregated UnWeighted Least Squares Support Recovery): the main idea is to take random subsets of the $N$ equations, perform a least squares recovery over this reduced bit of information and then average over many random subsets. We show that the proposed procedure can provably recover an approximation of $θ$ and demonstrate its use in support recovery through numerical examples.

math.ST↗

arXiv2020

Localized Quantitative Criteria for Equidistribution

Let $(x_n)_{n=1}^{\infty}$ be a sequence on the torus $\mathbb{T}$ (normalized to length 1). We show that if there exists a sequence of positive real numbers $(t_n)_{n=1}^{\infty}$ converging to 0 such that $$\lim_{N \rightarrow \infty}{ \frac{1}{N^2} \sum_{m,n = 1}^{N}{ \frac{1}{\sqrt{t_N}} \exp{\left(- \frac{1}{t_N} (x_m - x_n)^2 \right)}} } = \sqrtπ,$$ then $(x_n)_{n=1}^{\infty}$ is uniformly distributed. This is especially interesting when $t_N$ is close to $N^{-2}$ since the size of the sum is then mostly determined by local gaps at scale $\sim N^{-1}$. A similar argument can then be used to show equidistribution of sequences with Poissonian pair correlation, which recovers a recent result of Aistleitner, Lachmann \& Pausinger and Grepstad \& Larcher. The general form of the result is proven on arbitrary compact manifolds $(M,g)$ where the role of the exponential function is played by the heat kernel $e^{tΔ}$: for all $x_1, \dots, x_N \in M$ and all $t>0$ $$\frac{1}{N^2} \sum_{m,n=1}^N {[e^{tΔ}δ_{x_m}](x_n)} \geq \frac{1}{vol(M)}$$ and equality is attained as $N \rightarrow \infty$ if and only if $(x_n)_{n=1}^\infty$ equidistributes.

math.NT↗

arXiv2020

Fast localization of eigenfunctions via smoothed potentials

We study the problem of predicting highly localized low-lying eigenfunctions $(-Δ+V) ϕ= λϕ$ in bounded domains $Ω\subset \mathbb{R}^d$ for rapidly varying potentials $V$. Filoche & Mayboroda introduced the function $1/u$, where $(-Δ+ V)u=1$, as a suitable regularization of $V$ from whose minima one can predict the location of eigenfunctions with high accuracy. We proposed a fast method that produces a landscapes that is exceedingly similar, can be used for the same purposes and can be computed very efficiently: the computation time on an $n \times n$ grid, for example, is merely $\mathcal{O}(n^2 \log{n})$, the cost of two FFTs.

math.NA↗

arXiv2020

On a Kantorovich-Rubinstein inequality

An easy consequence of Kantorovich-Rubinstein duality is the following: if $f:[0,1]^d \rightarrow \infty$ is Lipschitz and $\left\{x_1, \dots, x_N \right\} \subset [0,1]^d$, then $$ \left| \int_{[0,1]^d} f(x) dx - \frac{1}{N} \sum_{k=1}^{N}{f(x_k)} \right| \leq \left\| \nabla f \right\|_{L^{\infty}} \cdot W_1\left( \frac{1}{N} \sum_{k=1}^{N}{δ_{x_k}} , dx\right),$$ where $W_1$ denotes the $1-$Wasserstein (or Earth Mover's) Distance. We prove another such inequality with a smaller norm on $\nabla f$ and a larger Wasserstein distance. Our inequality is sharp when the points are very regular, i.e. $W_{\infty} \sim N^{-1/d}$. This prompts the question whether these two inequalities are specific instances of an entire underlying family of estimates capturing a duality between transport distance and function space.

math.PR↗

arXiv2020

On the Wasserstein Distance between Classical Sequences and the Lebesgue Measure

We discuss the classical problem of measuring the regularity of distribution of sets of $N$ points in $\mathbb{T}^d$. A recent line of investigation is to study the cost ($=$ mass $\times$ distance) necessary to move Dirac measures placed in these points to the uniform distribution. We show that Kronecker sequences satisfy optimal transport distance in $d \geq 3$ dimensions. This shows that for differentiable $f: \mathbb{T}^d \rightarrow \mathbb{R}$ and badly approximable vectors $α\in \mathbb{R}^d$, we have $$ \ | \int_{\mathbb{T}^d} f(x) dx - \frac{1}{N} \sum_{k=1}^{N} f(k α) \ | \leq c_α \frac{ \| \nabla f\|^{(d-1)/d}_{L^{\infty}}\| \nabla f\|^{1/d}_{L^{2}} }{N^{1/d}}.$$ We note that the result is uniformly true for a sequence instead of a set. Simultaneously, it refines the classical integration error for Lipschitz functions, $\| \nabla f\|_{L^{\infty}} N^{-1/d}$. We obtain a similar improvement for numerical integration with respect to the regular grid. The main ingredient is an estimate involving Fourier coefficients of a measure; this allows for existing estimates to be conviently `recycled'. We present several open problems.

math.CA↗

arXiv2020

Conservation Laws for the Density of Roots of Polynomials under Differentiation

Let $p_n(x)$ be a polynomial of degree $n$ having $n$ distinct, real roots distributed according to a nice probability distribution $u(0,x)dx$ on $\mathbb{R}$. One natural problem is to understand the density $u(t,x)$ of the roots of the $(t\cdot n)-$th derivative of $p_n$ where $0 < t < 1$ as $n \rightarrow \infty$. We derive an \textit{infinite} number of conversation laws for the evolution of $u(t,x)$. The first three are \begin{align*} \int_{\mathbb{R}}{ u(t,x) ~ dx} = 1-t, \qquad \qquad \int_{\mathbb{R}}{ u(t,x) x ~ dx} = \left(1-t\right)\int_{\mathbb{R}}{ u(0,x) x~ dx}, \qquad \int_{\mathbb{R}} \int_{\mathbb{R}} u(t,x) (x-y)^2 u(t,y) ~ dx dy = (1-t)^3 \int_{\mathbb{R}} \int_{\mathbb{R}} u(0,x) (x-y)^2 u(0,y) ~ dx dy. \end{align*} The author suggested that $u(t,x)$ might evolve according to a nonlocal evolution equation involving the Hilbert transform; this has been verified for two special closed form solutions -- these conservation laws thus point to interesting identities for the Hilbert transform. We discuss many open problems.

math.AP↗

arXiv2020

Wasserstein Distance, Fourier Series and Applications

We study the Wasserstein metric $W_p$, a notion of distance between two probability distributions, from the perspective of Fourier Analysis and discuss applications. In particular, we bound the Earth Mover Distance $W_1$ between the distribution of quadratic residues in a finite field $\mathbb{F}_p$ and uniform distribution by $\lesssim p^{-1/2}$ (the Polya-Vinogradov inequality implies $\lesssim p^{-1/2} \log{p}$). We also show for continuous $f:\mathbb{T} \rightarrow \mathbb{R}_{}$ with mean value 0 $$ (\mbox{number of roots of}~f) \cdot \left( \sum_{k=1}^{\infty}{ \frac{ |\hat{f}(k)|^2}{k^2}}\right)^{\frac{1}{2}} \gtrsim \frac{\|f\|^{2}_{L^1(\mathbb{T})}}{\|f\|_{L^{\infty}(\mathbb{T})}}.$$ Moreover, we show that for a Laplacian eigenfunction $-Δ_g ϕ_λ = λϕ_λ$ on a compact Riemannian manifold $W_p\left(\max\left\{ϕ_λ, 0\right\}dx, \max\left\{-ϕ_λ, 0\right\} dx\right) \lesssim_p \sqrt{\logλ/λ} \|ϕ_λ\|_{L^1}^{1/p}$ which is at most a factor $\sqrt{\logλ}$ away from sharp. Several other problems are discussed.

math.CA↗

arXiv2020

On the Regularization Effect of Stochastic Gradient Descent applied to Least Squares

We study the behavior of stochastic gradient descent applied to $\|Ax -b \|_2^2 \rightarrow \min$ for invertible $A \in \mathbb{R}^{n \times n}$. We show that there is an explicit constant $c_{A}$ depending (mildly) on $A$ such that $$ \mathbb{E} ~\left\| Ax_{k+1}-b\right\|^2_{2} \leq \left(1 + \frac{c_{A}}{\|A\|_F^2}\right) \left\|A x_k -b \right\|^2_{2} - \frac{2}{\|A\|_F^2} \left\|A^T A (x_k - x)\right\|^2_{2}.$$ This is a curious inequality: the last term has one more matrix applied to the residual $u_k - u$ than the remaining terms: if $x_k - x$ is mainly comprised of large singular vectors, stochastic gradient descent leads to a quick regularization. For symmetric matrices, this inequality has an extension to higher-order Sobolev spaces. This explains a (known) regularization phenomenon: an energy cascade from large singular values to small singular values smoothes.

math.NA↗

arXiv2020

A Nonlocal Transport Equation Modeling Complex Roots of Polynomials under Differentiation

Let $p_n:\mathbb{C} \rightarrow \mathbb{C}$ be a random complex polynomial whose roots are sampled i.i.d. from a radial distribution $u(r) r dr$ in the complex plane. A natural question is how the distribution of roots evolves under repeated (say $n/2-$times) differentiation of the polynomial. We conjecture a mean-field expansion for the evolution of $ψ(s) = u(s) s$ $$ \frac{\partial ψ}{\partial t} = \frac{\partial}{\partial x} \left( \left( \frac{1}{x} \int_{0}^{x} ψ(s) ds \right)^{-1} ψ(x) \right).$$ The evolution of $ψ(s) \equiv 1$ corresponds to the evolution of random Taylor polynomials $$ p_n(z) = \sum_{k=0}^{n}{ γ_k \frac{z^k}{k!}} \quad \mbox{where} \quad γ_k \sim \mathcal{N}_{\mathbb{C}}(0,1).$$ We discuss some numerical examples suggesting that this particular solution may be stable. We prove that the solution is linearly stable. The linear stability analysis reduces to the classical Hardy integral inequality. Many open problems are discussed.

math.AP↗