SearcharxivSearch

arXiv subjects

Emily J. King

Publications and source records attributed to Emily J. King.

At least 19 recordsLinked to original sources

Taylor Diagram and Wasserstein Distance for Model Evaluation

The Taylor diagram is used to evaluate and compare predictive models with observed data and has many applications in climate and environmental sciences. Three statistics of interest--the centered root-mean-squared error, standard deviation, and the product-moment correlation coefficient--are related by the law of cosines; Taylor diagrams leverage this fact to allow visualization of all three statistics in a two-dimensional plot without any loss of information. In this work, we present a novel model evaluation tool--the Wasserstein-Taylor diagram--formed from integrating the Wasserstein distance from optimal transport into a Taylor diagram framework. This tool is built upon the fact that the law of cosines still holds in the Taylor diagram if the centered root-mean-squared error and product-moment correlation coefficient are replaced by the centered 2-Wasserstein distance and quantile correlation coefficient, respectively. The advantage of this Wasserstein-Taylor diagram is that one can use the distribution perspective afforded by the Wasserstein distance to compare observed and predicted data sets of different sizes and even to compare statistical models with only observed data. We further show that the quantile correlation coefficient on empirical measures converges to the quantile correlation coefficient on the sampled measures.

stat.ME

Towards a mathematical theory of superposition

We develop a mathematical theory of superposition in neural networks using tools from frame theory and compressed sensing. In our model, a sparse binary vector \(x\) of active features is encoded through an overcomplete dictionary \(W\), and feature recovery is performed by applying \(\operatorname{ReLU}(W^\top W x+b)\) with an appropriate bias vector \(b\). We prove several recovery theorems for this model. In the random-support setting, we establish high-probability support recovery for nearly tight, low-coherence dictionaries, with guarantees when the expected sparsity is up to order \(d/\log n\). In the worst-case support setting, we give a sharp and computable criterion for which sparsity levels permit support recovery. We apply this criterion to Gaussian random matrices and equiangular tight frames. For real equiangular tight frames with \(n>d+1\), we determine the exact recovery threshold in terms of the coherence. The proof of this result for real equiangular tight frames relies on a novel characterization---which should be of independent interest to frame theorists---of the distribution of signs in the Gram matrix.

stat.ML

Short spherical $t$-design curves

We study the minimum arclength of spherical $t$-design curves, i.e., closed rectifiable curves on $S^d$ whose normalized arclength measure exactly integrates every polynomial of degree at most $t$. We prove an explicit spectral lower bound that is sharp for $t=1$ in all spheres and for $t=2$ in every odd-dimensional sphere, yielding the first exact optimality results for spherical $t$-design curves with $t>1$. For even-dimensional spheres, we construct $2$-design curves whose lengths asymptotically match the lower bound as $d\to\infty$, and in $S^2$, we use numerical optimization and the calculus of variations to derive a candidate for the shortest $2$-design curve.

math.CO

Knowledge-Guided Machine Learning: Illustrating the use of Explainable Boosting Machines to Identify Overshooting Tops in Satellite Imagery

Machine learning (ML) algorithms have emerged in many meteorological applications. However, these algorithms struggle to extrapolate beyond the data they were trained on, i.e., they may adopt faulty strategies that lead to catastrophic failures. These failures are difficult to predict due to the opaque nature of ML algorithms. In high-stakes applications, such as severe weather forecasting, is is crucial to avoid such failures. One approach to address this issue is to develop more interpretable ML algorithms. The primary goal of this work is to illustrate the use of a specific interpretable ML algorithm that has not yet found much use in meteorology, Explainable Boosting Machines (EBMs). We demonstrate that EBMs are particularly suitable to implement human-guided strategies in an ML algorithm. As guiding example, we show how to develop an EBM to detect overshooting tops (OTs) in satellite imagery. EBMs require input features to be scalar. We use techniques from Knowledge-Guided Machine Learning to first extract scalar features from meteorological imagery. For the application of identifying OTs this includes extracting cloud texture from satellite imagery using Gray-Level Co-occurrence Matrices. Once trained, the EBM was examined and minimally altered to more closely match strategies used by domain scientists to identify OTs. The result of our efforts is a fully interpretable ML algorithm developed in a human-machine collaboration that uses human-guided strategies. While the final model does not reach the accuracy of more complex approaches, it performs reasonably well and we hope paves the way for building more interpretable ML algorithms for this and other meteorological applications.

cs.CV

A Land of Oblique Duality for Frames and Probabilistic Frames

Functions or distributions used to sample and to reconstruct signals often occur in different domains, like the Dirac delta and a band-limited bump function in classical sampling. Oblique dual frames generalize this phenomenon. In this paper, we provide new tools to study oblique dual frames and introduce a probabilistic variant of oblique dual frames. We first present the oblique dual frame potential and show that it is minimized precisely when the oblique dual coincides with the canonical oblique dual. We then define oblique dual probabilistic frames and oblique approximately dual probabilistic frames. In particular, we prove that for a given oblique dual probabilistic frame, the associated oblique dual probabilistic frame potential is minimized if and only if the frame is tight and the oblique dual is canonical. Moreover, the tightness assumption can be removed when the minimization is restricted to oblique dual probabilistic frames of pushforward type. Finally, we investigate perturbations of oblique dual probabilistic frames and show that if a probability measure is sufficiently close to an oblique dual probabilistic frame pair in the $2$-Wasserstein topology, then it forms an oblique approximately dual probabilistic frame.

math.FA

Testing isomorphism between tuples of subspaces

Given two tuples of subspaces, can you tell whether the tuples are isomorphic? We develop theory and algorithms to address this fundamental question. We focus on isomorphisms in which the ambient vector space is acted on by either a unitary group or general linear group. If isomorphism also allows permutations of the subspaces, then the problem is at least as hard as graph isomorphism. Otherwise, we provide a variety of polynomial-time algorithms with Matlab implementations to test for isomorphism. Keywords: subspace isomorphism, Grassmannian, Bargmann invariants, $H^\ast$-algebras, quivers, graph isomorphism

math.MG

$k$-Homogeneous Equiangular Tight Frames

We consider geometric and combinatorial characterizations of equiangular tight frames (ETFs), with the former concerning homogeneity of the vector and line symmetry groups and the latter the matroid structure. We introduce the concept of the bender of a frame, which is the collection of short circuits, which in turn are the dependent subsets of frame vectors of minimum size. We also show that ETFs with $k$-homogeneous line symmetry groups have benders which are $k$-designs. Paley ETFs are a known class of ETFs constructed using number theory. We determine the line and vector symmetry groups of all Paley ETFs and show that they are $2$-homogeneous. We additionally characterize all $k$-homogeneous ETFs for $k\geq 3$. Finally, we revisit David Larson's AMS Memoirs \emph{Frames, Bases, and Group Representations} coauthored with Deguang Han and \emph{Wandering Vectors for Unitary Systems and Orthogonal Wavelets} coauthored with Xingde Dai with a modern eye and focus on finite-dimensional Hilbert spaces.

math.FA

Approximately Dual and Pseudo-Dual Probabilistic Frames

This paper studies properties of dual probabilistic frames -- in particular in relation to redundancy -- and introduces both approximately dual probabilistic frames and pseudo-dual probabilistic frames. We show that the canonical dual probabilistic frame is the only dual frame of pushforward type of a probabilistic frame with zero redundancy. Furthermore, we show that probabilistic frames with finite redundancy are atomic and finite. Approximately dual probabilistic frames generalize duality, with pseudo-duality being a further generalization. We introduce these concepts and prove certain structural results. In particular, every probabilistic frame has a discrete finite frame as an approximate dual.

math.FA

Connections Between Frames with Rational Eigensteps and Semistandard Young Tableaux

In this paper, we explore a correspondence between frames with rational eigensteps and semistandard Young tableaux (SSYT), via the relation assigning a Gelfand-Tsetlin pattern to a frame via the frame's eigensteps. We will identify how certain key structures in SSYTs correlate with particular frame properties. For example, the weight of an SSYT yields the sequence of norms of any compatible frame. Additionally, this correspondence leads to a novel way to construct the eigensteps of a frame coming solely from tableaux. This is an alternative to the Top Kill algorithm which may be viewed as a combinatorial reinterpretation of the algorithm. We further employ other combinatorial techniques such as the boxcomp method to generate a ``complement" SSYT. On the frame side, this corresponds to a tight frame's Naimark complement as well as to a generalization of the Naimark complement for non-tight frames. Further research points to an analysis of equiangular tight frames and their corresponding tableaux, as well as using more combinatorial operations to further analyze frames.

math.CO

On the Structure of Frames and Equiangular Lines over Finite Fields and their Connections to Design Theory

This paper concerns frames and equiangular lines over finite fields. We find a necessary and sufficient condition for systems of equiangular lines over finite fields to be equiangular tight frames (ETFs). As is the case over subfields of $\mathbb{C}$, it is necessary for the Welch bound to be saturated, but there is an additional condition required involving sums of triple products. We also prove that similar to the case over $\mathbb{C}$, collections of vectors are similar to a regular simplex essentially when the triple products of their scalar products satisfy a certain property. Finally, we investigate switching equivalence classes of frames and systems of lines focusing on systems of equiangular lines in finite orthogonal geometries with maximal incoherent sets, drawing connections to combinatorial design theory.

math.CO

Dual Graphs of Polyhedral Decompositions for the Detection of Adversarial Attacks

Previous work has shown that a neural network with the rectified linear unit (ReLU) activation function leads to a convex polyhedral decomposition of the input space. These decompositions can be represented by a dual graph with vertices corresponding to polyhedra and edges corresponding to polyhedra sharing a facet, which is a subgraph of a Hamming graph. This paper illustrates how one can utilize the dual graph to detect and analyze adversarial attacks in the context of digital images. When an image passes through a network containing ReLU nodes, the firing or non-firing at a node can be encoded as a bit ($1$ for ReLU activation, $0$ for ReLU non-activation). The sequence of all bit activations identifies the image with a bit vector, which identifies it with a polyhedron in the decomposition and, in turn, identifies it with a vertex in the dual graph. We identify ReLU bits that are discriminators between non-adversarial and adversarial images and examine how well collections of these discriminators can ensemble vote to build an adversarial image detector. Specifically, we examine the similarities and differences of ReLU bit vectors for adversarial images, and their non-adversarial counterparts, using a pre-trained ResNet-50 architecture. While this paper focuses on adversarial digital images, ResNet-50 architecture, and the ReLU activation function, our methods extend to other network architectures, activation functions, and types of datasets.

cs.CV

A Primer on Topological Data Analysis to Support Image Analysis Tasks in Environmental Science

Topological data analysis (TDA) is a tool from data science and mathematics that is beginning to make waves in environmental science. In this work, we seek to provide an intuitive and understandable introduction to a tool from TDA that is particularly useful for the analysis of imagery, namely persistent homology. We briefly discuss the theoretical background but focus primarily on understanding the output of this tool and discussing what information it can glean. To this end, we frame our discussion around a guiding example of classifying satellite images from the Sugar, Fish, Flower, and Gravel Dataset produced for the study of mesocale organization of clouds by Rasp et. al. in 2020 (arXiv:1906:01906). We demonstrate how persistent homology and its vectorization, persistence landscapes, can be used in a workflow with a simple machine learning algorithm to obtain good results, and explore in detail how we can explain this behavior in terms of image-level features. One of the core strengths of persistent homology is how interpretable it can be, so throughout this paper we discuss not just the patterns we find, but why those results are to be expected given what we know about the theory of persistent homology. Our goal is that a reader of this paper will leave with a better understanding of TDA and persistent homology, be able to identify problems and datasets of their own for which persistent homology could be helpful, and gain an understanding of results they obtain from applying the included GitHub example code.

cs.LG

Formulating Beurling LASSO for Source Separation via Proximal Gradient Iteration

Beurling LASSO generalizes the LASSO problem to finite Radon measures regularized via their total variation. Despite its theoretical appeal, this space is hard to parametrize, which poses an algorithmic challenge. We propose a formulation of continuous convolutional source separation with Beurling LASSO that avoids the explicit computation of the measures and instead employs the duality transform of the proximal mapping.

eess.SP

Blind Source Separation in Polyphonic Music Recordings Using Deep Neural Networks Trained via Policy Gradients

We propose a method for the blind separation of sounds of musical instruments in audio signals. We describe the individual tones via a parametric model, training a dictionary to capture the relative amplitudes of the harmonics. The model parameters are predicted via a U-Net, which is a type of deep neural network. The network is trained without ground truth information, based on the difference between the model prediction and the individual time frames of the short-time Fourier transform. Since some of the model parameters do not yield a useful backpropagation gradient, we model them stochastically and employ the policy gradient instead. To provide phase information and account for inaccuracies in the dictionary-based representation, we also let the network output a direct prediction, which we then use to resynthesize the audio signals for the individual instruments. Due to the flexibility of the neural network, inharmonicity can be incorporated seamlessly and no preprocessing of the input spectra is required. Our algorithm yields high-quality separation results with particularly low interference on a variety of different audio samples, both acoustic and synthetic, provided that the sample contains enough data for the training and that the spectral characteristics of the musical instruments are sufficiently stable to be approximated by the dictionary.

eess.AS

Edge, Ridge, and Blob Detection with Symmetric Molecules

We present a novel approach to the detection and characterization of edges, ridges, and blobs in two-dimensional images which exploits the symmetry properties of directionally sensitive analyzing functions in multiscale systems that are constructed in the framework of alpha-molecules. The proposed feature detectors are inspired by the notion of phase congruency, stable in the presence of noise, and by definition invariant to changes in contrast. We also show how the behavior of coefficients corresponding to differently scaled and oriented analyzing functions can be used to obtain a comprehensive characterization of the geometry of features in terms of local tangent directions, widths, and heights. The accuracy and robustness of the proposed measures are validated and compared to various state-of-the-art algorithms in extensive numerical experiments in which we consider sets of clean and distorted synthetic images that are associated with reliable ground truths. To further demonstrate the applicability, we show how the proposed ridge measure can be used to detect and characterize blood vessels in digital retinal images and how the proposed blob measure can be applied to automatically count the number of cell colonies in a Petri dish.

cs.CV

Grassmannian codes from paired difference sets

An equiangular tight frame (ETF) is a sequence of vectors in a Hilbert space that achieves equality in the Welch bound and so has minimal coherence. More generally, an equichordal tight fusion frame (ECTFF) is a sequence of equi-dimensional subspaces of a Hilbert space that achieves equality in Conway, Hardin and Sloane's simplex bound. Every ECTFF is a type of optimal Grassmannian code, that is, an optimal packing of equi-dimensional subspaces of a Hilbert space. We construct ECTFFs by exploiting new relationships between known ETFs. Harmonic ETFs equate to difference sets for finite abelian groups. We say that a difference set for such a group is "paired" with a difference set for its Pontryagin dual when the corresponding subsequence of its harmonic ETF happens to be an ETF for its span. We show that every such pair yields an ECTFF. We moreover construct an infinite family of paired difference sets using quadratic forms over the field of two elements. Together this yields two infinite families of real ECTFFs.

math.FA