arXiv subjects

Weilin Li

Publications and source records attributed to Weilin Li.

30 records · Page 2Linked to original sources

arXiv2021

Maximal function pooling with applications

Inspired by the Hardy-Littlewood maximal function, we propose a novel pooling strategy which is called maxfun pooling. It is presented both as a viable alternative to some of the most popular pooling functions, such as max pooling and average pooling, and as a way of interpolating between these two algorithms. We demonstrate the features of maxfun pooling with two applications: first in the context of convolutional sparse coding, and then for image classification.

cs.CV↗

arXiv2021

Exploring the high dimensional geometry of HSI features

We explore feature space geometries induced by the 3-D Fourier scattering transform and deep neural network with extended attribute profiles on four standard hyperspectral images. We examine the distances and angles of class means, the variability of classes, and their low-dimensional structures. These statistics are compared to that of raw features, and our results provide insight into the vastly different properties of these two methods. We also explore a connection with the newly observed deep learning phenomenon of neural collapse.

cs.CV↗

arXiv2021

Generalization error of minimum weighted norm and kernel interpolation

We study the generalization error of functions that interpolate prescribed data points and are selected by minimizing a weighted norm. Under natural and general conditions, we prove that both the interpolants and their generalization errors converge as the number of parameters grow, and the limiting interpolant belongs to a reproducing kernel Hilbert space. This rigorously establishes an implicit bias of minimum weighted norm interpolation and explains why norm minimization may either benefit or suffer from over-parameterization. As special cases of this theory, we study interpolation by trigonometric polynomials and spherical harmonics. Our approach is from a deterministic and approximation theory viewpoint, as opposed to a statistical or random matrix one.

math.NA↗

arXiv2020

Three-Dimensional Fourier Scattering Transform and Classification of Hyperspectral Images

Recent developments in machine learning and signal processing have resulted in many new techniques that are able to effectively capture the intrinsic yet complex properties of hyperspectral imagery. Tasks ranging from anomaly detection to classification can now be solved by taking advantage of very efficient algorithms which have their roots in representation theory and in computational approximation. Time-frequency methods are one example of such techniques. They provide means to analyze and extract the spectral content from data. On the other hand, hierarchical methods such as neural networks incorporate spatial information across scales and model multiple levels of dependencies between spectral features. Both of these approaches have recently been proven to provide significant advances in the spectral-spatial classification of hyperspectral imagery. The 3D Fourier scattering transform, which is introduced in this paper, is an amalgamation of time-frequency representations with neural network architectures. It leverages the benefits provided by the Short-Time Fourier Transform with the numerical efficiency of deep learning network structures. We test the proposed method on several standard hyperspectral datasets, and we present results that indicate that the 3D Fourier scattering transform is highly effective at representing spectral content when compared with other state-of-the-art spectral-spatial classification methods.

cs.CV↗

arXiv2019

Super-resolution limit of the ESPRIT algorithm

The problem of imaging point objects can be formulated as estimation of an unknown atomic measure from its $M+1$ consecutive noisy Fourier coefficients. The standard resolution of this inverse problem is $1/M$ and super-resolution refers to the capability of resolving atoms at a higher resolution. When any two atoms are less than $1/M$ apart, this recovery problem is highly challenging and many existing algorithms either cannot deal with this situation or require restrictive assumptions on the sign of the measure. ESPRIT is an efficient method that does not depend on the sign of the measure. This paper provides an explicit error bound on the support matching distance of ESPRIT in terms of the minimum singular value of Vandermonde matrices. When the support consists of multiple well-separated clumps and noise is sufficiently small, the support error by ESPRIT scales like ${\rm SRF}^{2λ+2} \times {\rm Noise}$, where the Super-Resolution Factor (${\rm SRF}$) governs the difficulty of the problem and $λ$ is the cardinality of the largest clump. {If the support contains one clump of closely spaced atoms, the min-max error is ${\rm SRF}^{2λ+2} \times {\rm Noise}/M$. Our error bound matches the min-max rate up to a factor of $M$ in the small noise regime. Our results therefore establishes the near-optimality of ESPRIT,} and our theory is validated by numerical experiments.

cs.IT↗

arXiv2019

High-performance quantization for spectral super-resolution

We show that the method of distributed noise-shaping beta-quantization offers superior performance for the problem of spectral super-resolution with quantization whenever there is redundancy in the number of measurements. More precisely, if the (integer) oversampling ratio $λ$ is such that $\lfloor M/λ\rfloor - 1\geq 4/Δ$, where $M$ denotes the number of Fourier measurements and $Δ$ is the minimum separation distance associated with the atomic measure to be resolved, then for any number $K\geq 2$ of quantization levels available for the real and imaginary parts of the measurements, our quantization method guarantees reconstruction accuracy of order $O(λK^{- λ/2})$, up to constants which are independent of $K$ and $λ$. In contrast, memoryless scalar quantization offers a guarantee of order $O(K^{-1})$ only.

cs.IT↗

arXiv2019

Conditioning of restricted Fourier matrices and super-resolution of MUSIC

This paper studies stable recovery of a collection of point sources from its noisy $M+1$ low-frequency Fourier coefficients. We focus on the super-resolution regime where the minimum separation of the point sources is below $1/M$. We propose a separated clumps model where point sources are clustered in far apart sets, and prove an accurate lower bound of the Fourier matrix with nodes restricted to the source locations. This estimate gives rise to a theoretical analysis on the super-resolution limit of the MUSIC algorithm.

cs.IT↗

arXiv2018

Analysis of time-frequency scattering transforms

In this paper we address the problem of constructing a feature extractor which combines Mallat's scattering transform framework with time-frequency (Gabor) representations. To do this, we introduce a class of frames, called uniform covering frames, which includes a variety of semi-discrete Gabor systems. Incorporating a uniform covering frame with a neural network structure yields the Fourier scattering transform $\mathcal{S}_\mathcal{F}$ and the truncated Fourier scattering transform. We prove that $\mathcal{S}_\mathcal{F}$ propagates energy along frequency decreasing paths and its energy decays exponentially as a function of the depth. These quantitative estimates are fundamental in showing that $\mathcal{S}_\mathcal{F}$ satisfies the typical scattering transform properties, and in controlling the information loss due to width and depth truncation. We introduce the fast Fourier scattering transform algorithm, and illustrate the algorithm's performance. The time-frequency covering techniques developed in this paper are flexible and give insight into the analysis of scattering transforms.

math.FA↗

arXiv2017

Rotationally invariant time-frequency scattering transforms

In this paper we construct directionally sensitive functions that can be viewed as directional time-frequency representations. We call such a sequence a rotational uniform covering frame and by studying rotations of the frame, we derive the rotational Fourier scattering transform and the truncated rotational Fourier scattering transform. We prove that both operators are rotationally invariant, are bounded above and below, are non-expansive, and contract small translations and additive diffeomorphisms. We also construct finite uniform covering frames.

math.FA↗

arXiv2017

Elementary $L^\infty$ error estimates for super-resolution de-noising

This paper studies the problem of recovering a discrete complex measure on the torus from a finite number of corrupted Fourier samples. We assume the support of the unknown discrete measure satisfies a minimum separation condition and we use convex regularization methods to recover approximations of the original measure. We focus on two well-known convex regularization methods, and for both, we establish an error estimate that bounds the smoothed-out error in terms of the target resolution and noise level. Our $L^\infty$ approximation rate is entirely new for one of the methods, and improves upon a previously established $L^1$ estimate for the other. We provide a unified analysis and an elementary proof of the theorem.

cs.IT↗

arXiv2016

Super-resolution by means of Beurling minimal extrapolation

Let $M(\mathbb{T}^d)$ be the space of complex bounded Radon measures defined on the $d$-dimensional torus group $(\mathbb{R}/\mathbb{Z})^d=\mathbb{T}^d$, equipped with the total variation norm $\|\cdot\|$; and let $\hatμ$ denote the Fourier transform of $μ\in M(\mathbb{T}^d)$. We address the super-resolution problem: For given spectral (Fourier transform) data defined on a finite set $Λ\subset\mathbb{Z}^d$, determine if there is a unique $μ\in M(\mathbb{T}^d)$ of minimal norm for which $\hatμ$ equals this data on $Λ$. Without additional assumptions on $μ$ and $Λ$, our main theorem shows that the solutions to the super-resolution problem, which we call minimal extrapolations, depend crucially on the set $Γ\subsetΛ$, defined in terms of $μ$ and $Λ$. For example, when $Γ=0$, the minimal extrapolations are singular measures supported in the zero set of an analytic function, and when $Γ\geq 2$, the minimal extrapolations are singular measures supported in the intersection of $Γ\choose 2$ hyperplanes. By theory and example, we show that the case $Γ=1$ is different from other cases and is deeply connected with the existence of positive minimal extrapolations. This theorem has implications to the possibility and impossibility of uniquely recovering $μ$ from $Λ$. We illustrate how to apply our theory to both directions, by computing pertinent analytical examples. These examples are of interest in both super-resolution and deterministic compressed sensing. Our concept of an admissibility range fundamentally connects Beurling's theory of minimal extrapolation with Candes and Fernandez-Granda's work on super-resolution. This connection is exploited to address situations where current algorithms fail to compute a numerical solution to the super-resolution problem.

math.FA↗

arXiv2014

Boundary Value Problems on a Half Sierpinski Gasket

We study boundary value problems for the Laplacian on a domain $Ω$ consisting of the left half of the Sierpinski Gasket ($SG$), whose boundary is essentially a countable set of points $X$. For harmonic functions we give an explicit Poisson integral formula to recover the function from its boundary values, and characterize those that correspond to functions of finite energy. We give an explicit Dirichlet to Neumann map and show that it is invertible. We give an explicit description of the Dirichlet to Neumann spectra of the Laplacian with an exact count of the dimensions of eigenspaces. We compute the exact trace spaces on $X$ of the $L^2$ and $L^\infty$ domains of the Laplacian on $SG$. In terms of the these trace spaces, we characterize the functions in the $L^2$ and $L^\infty$ domains of the Laplacian on $Ω$ that extend to the corresponding domains on $SG$, and give an explicit linear extension operator in terms of piecewise biharmonic functions.

math.AP↗