arXiv subjects

Philipp Grohs

Publications and source records attributed to Philipp Grohs.

70 records · Page 4Linked to original sources

arXiv2018

Energy Propagation in Deep Convolutional Neural Networks

Many practical machine learning tasks employ very deep convolutional neural networks. Such large depths pose formidable computational challenges in training and operating the network. It is therefore important to understand how fast the energy contained in the propagated signals (a.k.a. feature maps) decays across layers. In addition, it is desirable that the feature extractor generated by the network be informative in the sense of the only signal mapping to the all-zeros feature vector being the zero input signal. This "trivial null-set" property can be accomplished by asking for "energy conservation" in the sense of the energy in the feature vector being proportional to that of the corresponding input signal. This paper establishes conditions for energy conservation (and thus for a trivial null-set) for a wide class of deep convolutional neural network-based feature extractors and characterizes corresponding feature map energy decay rates. Specifically, we consider general scattering networks employing the modulus non-linearity and we find that under mild analyticity and high-pass conditions on the filters (which encompass, inter alia, various constructions of Weyl-Heisenberg filters, wavelets, ridgelets, ($α$)-curvelets, and shearlets) the feature map energy decays at least polynomially fast. For broad families of wavelets and Weyl-Heisenberg filters, the guaranteed decay rate is shown to be exponential. Moreover, we provide handy estimates of the number of layers needed to have at least $((1-\varepsilon)\cdot 100)\%$ of the input signal energy be contained in the feature vector.

cs.IT↗

arXiv2017

Anisotropic Multiscale Systems on Bounded Domains

We provide a construction of multiscale systems on a bounded domain $Ω\subset \mathbb{R}^2$ coined boundary shearlet systems, which satisfy several properties advantageous for applications to imaging science and the numerical analysis of partial differential equations. More precisely, we construct boundary shearlet systems that form frames for the Sobolev spaces $H^s(Ω),s\in \mathbb{N} \cup \{0\},$ with controllable frame bounds and admit optimally sparse approximations for functions, which are smooth apart from a curve-like discontinuity. We show that the constructed systems allow incorporating boundary conditions. Furthermore, for $s \geq 0$ and $f\in H^s(Ω)$ we prove that weighted $\ell^2$ norms of the $L^2-$analysis coefficients of $f$ are equivalent to its $H^s(Ω)$ norm. This yields in particular, that the reweighted systems are frames also for $H^{-s}(Ω)$. Moreover, we demonstrate numerically, that the associated $L^2-$synthesis operator is also stable as a map to $H^s(Ω)$ which, in combination with the previous result, strongly indicates that these systems constitute so-called Gelfand frames for $(H^s(Ω), L^2(Ω), H^{-s}(Ω))$.

math.FA↗

arXiv2017

Stable Gabor Phase Retrieval and Spectral Clustering

We consider the problem of reconstructing a signal $f$ from its spectrogram, i.e., the magnitudes $|V_φf|$ of its Gabor transform $$V_φf (x,y):=\int_{\mathbb{R}}f(t)e^{-π(t-x)^2}e^{-2πıy t}dt, \quad x,y\in \mathbb{R}.$$ Such problems occur in a wide range of applications, from optical imaging of nanoscale structures to audio processing and classification. While it is well-known that the solution of the above Gabor phase retrieval problem is unique up to natural identifications, the stability of the reconstruction has remained wide open. The present paper discovers a deep and surprising connection between phase retrieval, spectral clustering and spectral geometry. We show that the stability of the Gabor phase reconstruction is bounded by the reciprocal of the Cheeger constant of the flat metric on $\mathbb{R}^2$, conformally multiplied with $|V_φf|$. The Cheeger constant, in turn, plays a prominent role in the field of spectral clustering, and it precisely quantifies the `disconnectedness' of the measurements $V_φf$. It has long been known that a disconnected support of the measurements results in an instability -- our result for the first time provides a converse in the sense that there are no other sources of instabilities. Due to the fundamental importance of Gabor phase retrieval in coherent diffraction imaging, we also provide a new understanding of the stability properties of these imaging techniques: Contrary to most classical problems in imaging science whose regularization requires the promotion of smoothness or sparsity, the correct regularization of the phase retrieval problem promotes the `connectedness' of the measurements in terms of bounding the Cheeger constant from below. Our work thus, for the first time, opens the door to the development of efficient regularization strategies.

math.FA↗

arXiv2017

Stable Phase Retrieval in Infinite Dimensions

The problem of phase retrieval is to determine a signal $f\in \mathcal{H}$, with $\mathcal{H}$ a Hilbert space, from intensity measurements $|F(ω)|$, where $F(ω):=\langle f , φ_ω\rangle$ are measurements of $f$ with respect to a measurement system $(φ_ω)_{ω\in Ω}\subset \mathcal{H}$. Although phase retrieval is always stable in the finite dimensional setting whenever it is possible (i.e. injectivity implies stability for the inverse problem), the situation is drastically different if $\mathcal{H}$ is infinite-dimensional: in that case phase retrieval is never uniformly stable [8, 4]; moreover the stability deteriorates severely in the dimension of the problem [8]. On the other hand, all empirically observed instabilities are of a certain type: they occur whenever the function $|F|$ of intensity measurements is concentrated on disjoint sets $D_j\subset Ω$, i.e., when $F= \sum_{j=1}^k F_j$ where each $F_j$ is concentrated on $D_j$ (and $k \geq 2$). Motivated by these considerations we propose a new paradigm for stable phase retrieval by considering the problem of reconstructing $F$ up to a phase factor that is not global, but that can be different for each of the subsets $D_j$, i.e., recovering $F$ up to the equivalence $$ F \sim \sum_{j=1}^k e^{i α_j} F_j.$$ We present concrete applications (for example in audio processing) where this new notion of stability is natural and meaningful and show that in this setting stable phase retrieval can actually be achieved, for instance if the measurement system is a Gabor frame or a frame of Cauchy wavelets.

math.FA↗

arXiv2016

Phase retrieval in the general setting of continuous frames for Banach spaces

We develop a novel and unifying setting for phase retrieval problems that works in Banach spaces and for continuous frames and consider the questions of uniqueness and stability of the reconstruction from phaseless measurements. Our main result states that also in this framework, the problem of phase retrieval is never uniformly stable in infinite dimensions. On the other hand, we show weak stability of the problem. This complements recent work [9], where it has been shown that phase retrieval is always unstable for the setting of discrete frames in Hilbert spaces. In particular, our result implies that the stability properties cannot be improved by oversampling the underlying discrete frame. We generalize the notion of complement property (CP) to the setting of continuous frames for Banach spaces (over $\mathbb{K}=\mathbb{R}$ or $\mathbb{K}=\mathbb{C}$) and verify that it is a necessary condition for uniqueness of the phase retrieval problem; when $\mathbb{K}=\mathbb{R}$ the CP is also sufficient for uniqueness. In our general setting, we also prove a conjecture posed by Bandeira et al. [5], which was originally formulated for finite-dimensional spaces: for the case $\mathbb{K}=\mathbb{C}$ the strong complement property (SCP) is a necessary condition for stability. To prove our main result, we show that the SCP can never hold for frames of infinite-dimensional Banach spaces.

math.FA↗

arXiv2016

Reconstructing real-valued functions from unsigned coefficients with respect to wavelet and other frames

In this paper we consider the following problem of phase retrieval: Given a collection of real-valued band-limited functions $\{ψ_λ\}_{λ\in Λ}\subset L^2(\mathbb{R}^d)$ that constitutes a semi-discrete frame, we ask whether any real-valued function $f \in L^2(\mathbb{R}^d)$ can be uniquely recovered from its unsigned convolutions ${\{|f \ast ψ_λ|\}_{λ\in Λ}}$. We find that under some mild assumptions on the semi-discrete frame and if $f$ has exponential decay at $\infty$, it suffices to know $|f \ast ψ_λ|$ on suitably fine lattices to uniquely determine $f$ (up to a global sign factor). We further establish a local stability property of our reconstruction problem. Finally, for two concrete examples of a (discrete) frame of $L^2(\mathbb{R}^d)$, $d=1,2$, we show that through sufficient oversampling one obtains a frame such that any real-valued function with exponential decay can be uniquely recovered from its unsigned frame coefficients.

math.FA↗

arXiv2016

Discrete Deep Feature Extraction: A Theory and New Architectures

First steps towards a mathematical theory of deep convolutional neural networks for feature extraction were made---for the continuous-time case---in Mallat, 2012, and Wiatowski and Bölcskei, 2015. This paper considers the discrete case, introduces new convolutional neural network architectures, and proposes a mathematical framework for their analysis. Specifically, we establish deformation and translation sensitivity results of local and global nature, and we investigate how certain structural properties of the input signal are reflected in the corresponding feature vectors. Our theory applies to general filters and general Lipschitz-continuous non-linearities and pooling operators. Experiments on handwritten digit classification and facial landmark detection---including feature importance evaluation---complement the theoretical findings.

cs.LG↗

arXiv2016

On the Approximation of Functions with Line Singularities by Ridgelets

In [GO15], the authors discussed the existence of numerically feasible solvers for advection equations that run in optimal computational complexity. In this paper, we complete the last remaining requirement to achieve this goal - by showing that ridgelets, on which the solver is based, approximate functions with line singularities (which may appear as solutions to the advection equation) with the best possible approximation rate. Structurally, the proof resembles [Can01], where a similar result was proved for a different ridgelet construction, which is however not well-suited for use in a PDE solver (and in particular, not suitable for the CDD-schemes [CDD01] we are interested in). Due to the differences between the two ridgelet constructions, we have to deal with quite a different set of issues, but are also able to relax the (support) conditions on the function being approximated. Finally, the proof employs a new convolution-type estimate that could be of independent interest due to its sharpness.

math.NA↗

arXiv2014

Optimal Adaptive Ridgelet Schemes for Linear Transport Equations

In this paper we present a novel method for the numerical solution of linear transport equations, which is based on ridgelets. Such equations arise for instance in radiative transfer or in phase contrast imaging. Due to the fact that ridgelet systems are well adapted to the structure of linear transport operators, it can be shown that our scheme operates in optimal complexity, even if line singularities are present in the solution. The key to this is showing that the system matrix (with diagonal preconditioning) is uniformly well-conditioned and compressible -- the proof for the latter represents the main part of the paper. We conclude with some numerical experiments about $N$-term approximations and how they are recovered by the solver, as well as localisation of singularities in the ridgelet frame.

math.NA↗

arXiv2014

$α$-Molecules

Within the area of applied harmonic analysis, various multiscale systems such as wavelets, ridgelets, curvelets, and shearlets have been introduced and successfully applied. The key property of each of those systems are their (optimal) approximation properties in terms of the decay of the $L^2$-error of the best $N$-term approximation for a certain class of functions. In this paper, we introduce the general framework of $α$-molecules, which encompasses most multiscale systems from applied harmonic analysis, in particular, wavelets, ridgelets, curvelets, and shearlets as well as extensions of such with $α$ being a parameter measuring the degree of anisotropy, as a means to allow a unified treatment of approximation results within this area. Based on an $α$-scaled index distance, we first prove that two systems of $α$-molecules are almost orthogonal. This leads to a general methodology to transfer approximation results within this framework, provided that certain consistency and time-frequency localization conditions of the involved systems of $α$-molecules are satisfied. We finally utilize these results to enable the derivation of optimal sparse approximation results \msch{for} a specific class of cartoon-like functions by sufficient conditions on the 'control' parameters of a system of $α$-molecules.

math.FA↗

arXiv2014

Cartoon Approximation with $α$-Curvelets

It is well-known that curvelets provide optimal approximations for so-called cartoon images which are defined as piecewise $C^2$-functions, separated by a $C^2$ singularity curve. In this paper, we consider the more general case of piecewise $C^β$-functions, separated by a $C^β$ singularity curve for $β\in (1,2]$. We first prove a benchmark result for the possibly achievable best $N$-term approximation rate for this more general signal model. Then we introduce what we call $α$-curvelets, which are systems that interpolate between wavelet systems on the one hand ($α= 1$) and curvelet systems on the other hand ($α= \frac12$). Our main result states that those frames achieve this optimal rate for $α= \frac{1}β$, up to $\log$-factors.

math.FA↗

arXiv2014

Intrinsic Localization of Anisotropic Frames II: $α$-Molecules

This article is a continuation of the recent paper [Grohs, Intrinsic localization of anisotropic frames, ACHA, 2013], where off-diagonal-decay properties (often referred to as 'localization' in the literature) of Moore-Penrose pseudoinverses of (bi-infinite) matrices are established, whenever the latter possess similar off-diagonal-decay properties. This problem is especially interesting if the matrix arises as a discretization of an operator with respect to a frame or basis. Previous work on this problem has been restricted to wavelet- or Gabor frames. In the previous work we extended these results to frames of parabolic molecules, including curvelets or shearlets as special cases. The present paper extends and unifies these results by establishing analogous properties for frames of $α$-molecules as introduced in recent work [Grohs, Keiper, Kutyniok, Schäfer, Alpha molecules: curvelets, shearlets, ridgelets, and beyond, Proc. SPIE. 8858, 2013]. Since wavelets, curvelets, shearlets, ridgelets and hybrid shearlets all constitute instances of $α$-molecules, our results establish localization properties for all these systems simultaneously.

math.FA↗

arXiv2012

Parabolic Molecules

Anisotropic decompositions using representation systems based on parabolic scaling such as curvelets or shearlets have recently attracted significantly increased attention due to the fact that they were shown to provide optimally sparse approximations of functions exhibiting singularities on lower dimensional embedded manifolds. The literature now contains various direct proofs of this fact and of related sparse approximation results. However, it seems quite cumbersome to prove such a canon of results for each system separately, while many of the systems exhibit certain similarities. In this paper, with the introduction of the notion of {\em parabolic molecules}, we aim to provide a comprehensive framework which includes customarily employed representation systems based on parabolic scaling such as curvelets and shearlets. It is shown that pairs of parabolic molecules have the fundamental property to be almost orthogonal in a particular sense. This result is then applied to analyze parabolic molecules with respect to their ability to sparsely approximate data governed by anisotropic features. For this, the concept of {\em sparsity equivalence} is introduced which is shown to allow the identification of a large class of parabolic molecules providing the same sparse approximation results as curvelets and shearlets. Finally, as another application, smoothness spaces associated with parabolic molecules are introduced providing a general theoretical approach which even leads to novel results for, for instance, compactly supported shearlets.

math.FA↗

arXiv2010

Continuous Shearlet Tight Frames

Based on the shearlet transform we present a general construction of continuous tight frames for $L^2(\mathbb{R}^2)$ from any sufficiently smooth function with anisotropic moments. This includes for example compactly supported systems, piecewise polynomial systems, or both. From our earlier results it follows that these systems enjoy the same desirable approximation properties for directional data as the previous bandlimited and very specific constructions due to Kutyniok and Labate. We also show that the representation formulas we derive are in a sense optimal for the shearlet transform.

math.FA↗

arXiv2010

Definability and stability of multiscale decompositions for manifold-valued data

We discuss multiscale representations of discrete manifold-valued data. As it turns out that we cannot expect general manifold-analogues of biorthogonal wavelets to possess perfect reconstruction, we focus our attention on those constructions which are based on upscaling operators which are either interpolating or midpoint-interpolating. For definable multiscale decompositions we obtain a stability result.

math.DG↗

arXiv2009

Continuous Shearlet Frames and Resolution of the Wavefront Set

In recent years directional multiscale transformations like the curvelet- or shearlet transformation have gained considerable attention. The reason for this is that these transforms are - unlike more traditional transforms like wavelets - able to efficiently handle data with features along edges. The main result in [G. Kutyniok, D. Labate. Resolution of the Wavefront Set using continuous Shearlets, Trans. AMS 361 (2009), 2719-2754] confirming this property for shearlets is due to Kutyniok and Labate where it is shown that for very special functions $ψ$ with frequency support in a compact conical wegde the decay rate of the shearlet coefficients of a tempered distribution $f$ with respect to the shearlet $ψ$ can resolve the Wavefront Set of $f$. We demonstrate that the same result can be verified under much weaker assumptions on $ψ$, namely to possess sufficiently many anisotropic vanishing moments. We also show how to build frames for $L^2(\mathbb{R}^2)$ from any such function. To prove our statements we develop a new approach based on an adaption of the Radon transform to the shearlet structure.

math.FA↗