arXiv subjects

John Urschel

Publications and source records attributed to John Urschel.

26 records · Page 2Linked to original sources

arXiv2023

Estimating the matrix $p \rightarrow q$ norm

The matrix $p \rightarrow q$ norm is a fundamental quantity appearing in a variety of areas of mathematics. This quantity is known to be efficiently computable in only a few special cases. The best known algorithms for approximately computing this quantity with theoretical guarantees essentially consist of computing the $p\to q$ norm for $p,q$ where this quantity can be computed exactly or up to a constant, and applying interpolation. We analyze the matrix $2 \to q$ norm problem and provide an improved approximation algorithm via a simple argument involving the rows of a given matrix. For example, we improve the best-known $2\to 4$ norm approximation from $m^{1/8}$ to $m^{1/12}$. This insight for the $2\to q$ norm improves the best known $p \to q$ approximation algorithm for the region $p \le 2 \le q$, and leads to an overall improvement in the best-known approximation for $p \to q$ norms from $m^{25/128}$ to $m^{3 - 2 \sqrt{2}}$.

cs.DS↗

arXiv2023

Nodal decompositions of a symmetric matrix

Analyzing nodal domains is a way to discern the structure of eigenvectors of operators on a graph. We give a new definition extending the concept of nodal domains to arbitrary signed graphs, and therefore to arbitrary symmetric matrices. We show that for an arbitrary symmetric matrix, a positive fraction of eigenbases satisfy a generalized version of known nodal bounds for un-signed (that is classical) graphs. We do this through an explicit decomposition. Moreover, we show that with high probability, the number of nodal domains of a bulk eigenvector of the adjacency matrix of signed a Erdős-Rényi graph is $Ω(n/\log n)$ and $o(n)$.

math-ph↗

arXiv2023

Representing the Special Linear Group with Block Unitriangular Matrices

We prove that every element of the special linear group can be represented as the product of at most six block unitriangular matrices, and that there exist matrices for which six products are necessary, independent of indexing. We present an analogous result for the general linear group. These results serve as general statements regarding the representational power of alternating linear updates. The factorizations and lower bounds of this work immediately imply tight estimates on the expressive power of linear affine coupling blocks in machine learning.

math.NA↗

arXiv2021

Multidimensional Scaling: Approximation and Complexity

Metric Multidimensional scaling (MDS) is a classical method for generating meaningful (non-linear) low-dimensional embeddings of high-dimensional data. MDS has a long history in the statistics, machine learning, and graph drawing communities. In particular, the Kamada-Kawai force-directed graph drawing method is equivalent to MDS and is one of the most popular ways in practice to embed graphs into low dimensions. Despite its ubiquity, our theoretical understanding of MDS remains limited as its objective function is highly non-convex. In this paper, we prove that minimizing the Kamada-Kawai objective is NP-hard and give a provable approximation algorithm for optimizing it, which in particular is a PTAS on low-diameter graphs. We supplement this result with experiments suggesting possible connections between our greedy approximation algorithm and gradient-based methods.

cs.LG↗

arXiv2021

Maximum spread of graphs and bipartite graphs

Given any graph $G$, the (adjacency) spread of $G$ is the maximum absolute difference between any two eigenvalues of the adjacency matrix of $G$. In this paper, we resolve a pair of 20-year-old conjectures of Gregory, Hershkowitz, and Kirkland regarding the spread of graphs. The first states that for all positive integers $n$, the $n$-vertex graph $G$ that maximizes spread is the join of a clique and an independent set, with $\lfloor 2n/3 \rfloor$ and $\lceil n/3 \rceil$ vertices, respectively. Using techniques from the theory of graph limits and numerical analysis, we prove this claim for all $n$ sufficiently large. As an intermediate step, we prove an analogous result for a family of operators in the Hilbert space over $\mathscr{L}^2[0,1]$. The second conjecture claims that for any fixed $e\leq n^2/4$, if $G$ maximizes spread over all $n$-vertex graphs with $e$ edges, then $G$ is bipartite. We prove an asymptotic version of this conjecture. Furthermore, we exhibit an infinite family of counterexamples, which shows that our asymptotic solution is tight up to lower order error terms.

math.CO↗

arXiv2017

Maximum likelihood estimation of determinantal point processes

Determinantal point processes (DPPs) have wide-ranging applications in machine learning, where they are used to enforce the notion of diversity in subset selection problems. Many estimators have been proposed, but surprisingly the basic properties of the maximum likelihood estimator (MLE) have received little attention. The difficulty is that it is a non-concave maximization problem, and such functions are notoriously difficult to understand in high dimensions, despite their importance in modern machine learning. Here we study both the local and global geometry of the expected log-likelihood function. We prove several rates of convergence for the MLE and give a complete characterization of the case where these are parametric. We also exhibit a potential curse of dimensionality where the asymptotic variance of the MLE scales exponentially with the dimension of the problem. Moreover, we exhibit an exponential number of saddle points, and give evidence that these may be the only critical points.

math.ST↗

arXiv2017

Rates of estimation for determinantal point processes

Determinantal point processes (DPPs) have wide-ranging applications in machine learning, where they are used to enforce the notion of diversity in subset selection problems. Many estimators have been proposed, but surprisingly the basic properties of the maximum likelihood estimator (MLE) have received little attention. In this paper, we study the local geometry of the expected log-likelihood function to prove several rates of convergence for the MLE. We also give a complete characterization of the case where the MLE converges at a parametric rate. Even in the latter case, we also exhibit a potential curse of dimensionality where the asymptotic variance of the MLE is exponentially large in the dimension of the problem.

math.ST↗

arXiv2017

Learning Determinantal Point Processes with Moments and Cycles

Determinantal Point Processes (DPPs) are a family of probabilistic models that have a repulsive behavior, and lend themselves naturally to many tasks in machine learning where returning a diverse set of objects is important. While there are fast algorithms for sampling, marginalization and conditioning, much less is known about learning the parameters of a DPP. Our contribution is twofold: (i) we establish the optimal sample complexity achievable in this problem and show that it is governed by a natural parameter, which we call the \emph{cycle sparsity}; (ii) we propose a provably fast combinatorial algorithm that implements the method of moments efficiently and achieves optimal sample complexity. Finally, we give experimental results that confirm our theoretical findings.

math.ST↗