arXiv subjects

Ingrid Daubechies

Publications and source records attributed to Ingrid Daubechies.

At least 19 recordsLinked to original sources

arXiv2024

Studying Morphological Variation: Exploring the Shape Space in Evolutionary Anthropology

We present results of a long-term team collaboration of mathematicians and biologists. We focus on building a mathematical framework for the shape space constituted by a collection of homologous bones or teeth from many species. The biological application is to quantitative morphological understanding of the evolutionary history of primates in particular, and mammals more generally. Similar to the practice of biologists, we leverage the power of the whole collection for results that are more robust than can be obtained by only pairwise comparisons, using tools from differential geometry and machine learning. This paper concentrates on the mathematical framework. We review methods for comparing anatomical surfaces, discuss the problem of registration and alignment, and address the computation of different distances. Next, we cover broader questions related to cross-dataset landmark selection, shape segmentation, and shape classification analysis. This paper summarizes the work of many team members other than the authors; in this paper that unites (for the first time) all their results in one joint context, space restrictions prevent a full description of the mathematical details, which are thoroughly covered in the original articles. Although our application is to the study of anatomical surfaces, we believe our approach has much wider applicability.

math.NA↗

arXiv2024

signDNE: A python package for ariaDNE and its sign-oriented extension

A key challenge in evolutionary biology is to develop robust computational tools that can accurately analyze shape variations across diverse anatomical structures. The Dirichlet Normal Energy (DNE) is a shape complexity metric that addresses this by summarizing the local curvature of surfaces, particularly aiding the analytical studies and providing insights into evolutionary and functional adaptations. Building on the DNE concept, we introduce a Python-based implementation, designed to compute both the original DNE and a newly developed sign-oriented DNE metric. This Python package includes a user-friendly command line interface (CLI) and built-in visualization tools to facilitate the interpretation of the surface's local curvature properties. The addition of signDNE, which integrates the convexity and concavity of surfaces, enhances the tool's ability to identify fine-scale features across a broad range of biological structures. We validate the robustness of our method by comparing its performance with standard implementations on a dataset of triangular meshes with varying discrete representations. Additionally, we demonstrate its potential applications through visualization of the local curvature field (i.e., local curvature value over the surface) on various biological specimens, showing how it effectively captures complex biological features. In this paper, we offer a brief overview of the Python CLI for ease of use. Alongside the Python implementation, we have also updated the original MATLAB package to ensure consistent and accurate DNE computation across platforms. These improvements enhance the tool's flexibility, reduce sensitivity to sampling density and mesh quality, and support a more accurate interpretation of biological surface topography.

q-bio.QM↗

arXiv2022

PiPs: a Kernel-based Optimization Scheme for Analyzing Non-Stationary 1D Signals

This paper proposes a novel kernel-based optimization scheme to handle tasks in the analysis, e.g., signal spectral estimation and single-channel source separation of 1D non-stationary oscillatory data. The key insight of our optimization scheme for reconstructing the time-frequency information is that when a nonparametric regression is applied on some input values, the output regressed points would lie near the oscillatory pattern of the oscillatory 1D signal only if these input values are a good approximation of the ground-truth phase function. In this work, Gaussian Process (GP) is chosen to conduct this nonparametric regression: the oscillatory pattern is encoded as the Pattern-inducing Points (PiPs) which act as the training data points in the GP regression; while the targeted phase function is fed in to compute the correlation kernels, acting as the testing input. Better approximated phase function generates more precise kernels, thus resulting in smaller optimization loss error when comparing the kernel-based regression output with the original signals. To the best of our knowledge, this is the first algorithm that can satisfactorily handle fully non-stationary oscillatory data, close and crossover frequencies, and general oscillatory patterns. Even in the example of a signal {produced by slow variation in the parameters of a trigonometric expansion}, we show that PiPs admits competitive or better performance in terms of accuracy and robustness than existing state-of-the-art algorithms.

stat.ML↗

arXiv2022

Stable phase retrieval for infinite dimensional subspaces of $L_2(\mathbb{R})$

Phase retrieval is known to always be unstable when using a frame or continuous frame for an infinite dimensional Hilbert space. We consider a generalization of phase retrieval to the setting of subspaces of $L_2$ which coincides with using a continuous frame for phase retrieval when the subspace is the range of the analysis operator of a continuous frame. We then prove that there do exist infinite dimensional subspaces of $L_2$ where phase retrieval is stable. That is, we give a method for constructing an infinite dimensional subspace $Y\subseteq L_2$ such that there exists $C\geq 1$ so that $$\min\big(\big\|f-g\big\|_{L_2},\big\|f+g\big\|_{L_2}\big)\leq C \big\| |f|-|g| \big\|_{L_2} \qquad\textrm{ for all }f,g\in Y. $$ This construction also leads to new results on uniform stability of phase retrieval in finite dimensions. Our construction has a deterministic component and a random component. When using sub-Gaussian random variables we achieve phase retrieval with high probability and stability constant independent of the dimension $n$ when using $m$ on the order of $n$ random vectors. Without sub-Gaussian or any other higher moment assumptions, we are able to achieve phase retrieval with high probability and stability constant independent of the dimension $n$ when using $m$ on the order of $n\log(n)$ random vectors.

math.FA↗

arXiv2022

Diffusion Maps : Using the Semigroup Property for Parameter Tuning

Diffusion maps (DM) constitute a classic dimension reduction technique, for data lying on or close to a (relatively) low-dimensional manifold embedded in a much larger dimensional space. The DM procedure consists in constructing a spectral parametrization for the manifold from simulated random walks or diffusion paths on the data set. However, DM is hard to tune in practice. In particular, the task to set a diffusion time t when constructing the diffusion kernel matrix is critical. We address this problem by using the semigroup property of the diffusion operator. We propose a semigroup criterion for picking t. Experiments show that this principled approach is effective and robust.

stat.ML↗

arXiv2022

Mixed X-Ray Image Separation for Artworks with Concealed Designs

In this paper, we focus on X-ray images of paintings with concealed sub-surface designs (e.g., deriving from reuse of the painting support or revision of a composition by the artist), which include contributions from both the surface painting and the concealed features. In particular, we propose a self-supervised deep learning-based image separation approach that can be applied to the X-ray images from such paintings to separate them into two hypothetical X-ray images. One of these reconstructed images is related to the X-ray image of the concealed painting, while the second one contains only information related to the X-ray of the visible painting. The proposed separation network consists of two components: the analysis and the synthesis sub-networks. The analysis sub-network is based on learned coupled iterative shrinkage thresholding algorithms (LCISTA) designed using algorithm unrolling techniques, and the synthesis sub-network consists of several linear mappings. The learning algorithm operates in a totally self-supervised fashion without requiring a sample set that contains both the mixed X-ray images and the separated ones. The proposed method is demonstrated on a real painting with concealed content, Doña Isabel de Porcel by Francisco de Goya, to show its effectiveness.

cs.CV↗

arXiv2021

Neural Network Approximation of Refinable Functions

In the desire to quantify the success of neural networks in deep learning and other applications, there is a great interest in understanding which functions are efficiently approximated by the outputs of neural networks. By now, there exists a variety of results which show that a wide range of functions can be approximated with sometimes surprising accuracy by these outputs. For example, it is known that the set of functions that can be approximated with exponential accuracy (in terms of the number of parameters used) includes, on one hand, very smooth functions such as polynomials and analytic functions (see e.g. \cite{E,S,Y}) and, on the other hand, very rough functions such as the Weierstrass function (see e.g. \cite{EPGB,DDFHP}), which is nowhere differentiable. In this paper, we add to the latter class of rough functions by showing that it also includes refinable functions. Namely, we show that refinable functions are approximated by the outputs of deep ReLU networks with a fixed width and increasing depth with accuracy exponential in terms of their number of parameters. Our results apply to functions used in the standard construction of wavelets as well as to functions constructed via subdivision algorithms in Computer Aided Geometric Design.

cs.LG↗

arXiv2021

Robust and Resource Efficient Identification of Shallow Neural Networks by Fewest Samples

We address the structure identification and the uniform approximation of sums of ridge functions $f(x)=\sum_{i=1}^m g_i(a_i\cdot x)$ on ${\mathbb R}^d$, representing a general form of a shallow feed-forward neural network, from a small number of query samples. Higher order differentiation, as used in our constructive approximations, of sums of ridge functions or of their compositions, as in deeper neural network, yields a natural connection between neural network weight identification and tensor product decomposition identification. In the case of the shallowest feed-forward neural network, second order differentiation and tensors of order two (i.e., matrices) suffice as we prove in this paper. We use two sampling schemes to perform approximate differentiation - active sampling, where the sampling points are universal, actively, and randomly designed, and passive sampling, where sampling points were preselected at random from a distribution with known density. Based on multiple gathered approximated first and second order differentials, our general approximation strategy is developed as a sequence of algorithms to perform individual sub-tasks. We first perform an active subspace search by approximating the span of the weight vectors $a_1,\dots,a_m$. Then we use a straightforward substitution, which reduces the dimensionality of the problem from $d$ to $m$. The core of the construction is then the stable and efficient approximation of weights expressed in terms of rank-$1$ matrices $a_i \otimes a_i$, realized by formulating their individual identification as a suitable nonlinear program. We prove the successful identification by this program of weight vectors being close to orthonormal and we also show how we can costructively reduce to this case by a whitening procedure, without loss of any generality.

stat.ML↗

arXiv2020

Approximating the Riemannian Metric from Point Clouds via Manifold Moving Least Squares

The approximation of both geodesic distances and shortest paths on point cloud sampled from an embedded submanifold $\mathcal{M}$ of Euclidean space has been a long-standing challenge in computational geometry. Given a sampling resolution parameter $ h $, state-of-the-art discrete methods yield $ O(h) $ provable approximations. In this paper, we investigate the convergence of such approximations made by Manifold Moving Least-Squares (Manifold-MLS), a method that constructs an approximating manifold $\mathcal{M}^h$ using information from a given point cloud that was developed by Sober \& Levin in 2019. In this paper, we show that provided that $\mathcal{M}\in C^{k}$ and closed (i.e. $\mathcal{M}$ is a compact manifold without boundary) the Riemannian metric of $ \mathcal{M}^h $ approximates the Riemannian metric of $ \mathcal{M}, $. Explicitly, given points $ p_1, p_2 \in \mathcal{M}$ with geodesic distance $ ρ_{\mathcal{M}}(p_1, p_2) $, we show that their corresponding points $ p_1^h, p_2^h \in \mathcal{M}^h$ have a geodesic distance of $ ρ_{\mathcal{M}^h}(p_1^h,p_2^h) = ρ_{\mathcal{M}}(p_1, p_2)(1 + O(h^{k-1})) $ (i.e., the Manifold-MLS is nearly an isometry). We then use this result, as well as the fact that $ \mathcal{M}^h $ can be sampled with any desired resolution, to devise a naive algorithm that yields approximate geodesic distances with a rate of convergence $ O(h^{k-1}) $. We show the potential and the robustness to noise of the proposed method on some numerical simulations.

math.NA↗

arXiv2020

Image Separation with Side Information: A Connected Auto-Encoders Based Approach

X-radiography (X-ray imaging) is a widely used imaging technique in art investigation. It can provide information about the condition of a painting as well as insights into an artist's techniques and working methods, often revealing hidden information invisible to the naked eye. In this paper, we deal with the problem of separating mixed X-ray images originating from the radiography of double-sided paintings. Using the visible color images (RGB images) from each side of the painting, we propose a new Neural Network architecture, based upon 'connected' auto-encoders, designed to separate the mixed X-ray image into two simulated X-ray images corresponding to each side. In this proposed architecture, the convolutional auto encoders extract features from the RGB images. These features are then used to (1) reproduce both of the original RGB images, (2) reconstruct the hypothetical separated X-ray images, and (3) regenerate the mixed X-ray image. The algorithm operates in a totally self-supervised fashion without requiring a sample set that contains both the mixed X-ray images and the separated ones. The methodology was tested on images from the double-sided wing panels of the \textsl{Ghent Altarpiece}, painted in 1432 by the brothers Hubert and Jan van Eyck. These tests show that the proposed approach outperforms other state-of-the-art X-ray image separation methods for art investigation applications.

eess.IV↗

arXiv2020

Stable Phase Retrieval from Locally Stable and Conditionally Connected Measurements

This paper is concerned with stable phase retrieval for a family of phase retrieval models we name "locally stable and conditionally connected" (LSCC) measurement schemes. For every signal $f$, we associate a corresponding weighted graph $G_f$, defined by the LSCC measurement scheme, and show that the phase retrievability of the signal $f$ is determined by the connectivity of $G_f$. We then characterize the phase retrieval stability of the signal $f$ by two measures that are commonly used in graph theory to quantify graph connectivity: the Cheeger constant of $G_f$ for real valued signals, and the algebraic connectivity of $G_f$ for complex valued signals. We use our results to study the stability of two phase retrieval models that can be cast as LSCC measurement schemes, and focus on understanding for which signals the "curse of dimensionality" can be avoided. The first model we discuss is a finite-dimensional model for locally supported measurements such as the windowed Fourier transform. For signals "without large holes", we show the stability constant exhibits only a mild polynomial growth in the dimension, in stark contrast with the exponential growth which uniform stability constants tend to suffer from; more precisely, in $R^d$ the constant grows proportionally to $d^{1/2}$, while in $C^d$ it grows proportionally to $d$. We also show the growth of the constant in the complex case cannot be reduced, suggesting that complex phase retrieval is substantially more difficult than real phase retrieval. The second model we consider is an infinite-dimensional phase retrieval problem in a principal shift invariant space. We show that despite the infinite dimensionality of this model, signals with monotone exponential decay will have a finite stability constant. In contrast, the stability bound provided by our results will be infinite if the signal's decay is polynomial.

cs.IT↗

arXiv2020

Non-Convex Planar Harmonic Maps

We formulate a novel characterization of a family of invertible maps between two-dimensional domains. Our work follows two classic results: The Radó-Kneser-Choquet (RKC) theorem, which establishes the invertibility of harmonic maps into a convex planer domain; and Tutte's embedding theorem for planar graphs - RKC's discrete counterpart - which proves the invertibility of piecewise linear maps of triangulated domains satisfying a discrete-harmonic principle, into a convex planar polygon. In both theorems, the convexity of the target domain is essential for ensuring invertibility. We extend these characterizations, in both the continuous and discrete cases, by replacing convexity with a less restrictive condition. In the continuous case, Alessandrini and Nesi provide a characterization of invertible harmonic maps into non-convex domains with a smooth boundary by adding additional conditions on orientation preservation along the boundary. We extend their results by defining a condition on the normal derivatives along the boundary, which we call the cone condition; this condition is tractable and geometrically intuitive, encoding a weak notion of local invertibility. The cone condition enables us to extend Alessandrini and Nesi to the case of harmonic maps into non-convex domains with a piecewise-smooth boundary. In the discrete case, we use an analog of the cone condition to characterize invertible discrete-harmonic piecewise-linear maps of triangulations. This gives an analog of our continuous results and characterizes invertible discrete-harmonic maps in terms of the orientation of triangles incident on the boundary.

math.AP↗

arXiv2019

Constructing curvelet-like bases and low-redundancy frames

We provide a detailed analysis of the obstruction (studied first by S. Durand and later by R. Yin and one of us) in the construction of multidirectional wavelet orthonormal bases corresponding to any admissible frequency partition in the framework of subband filtering with non-uniform subsampling. To contextualize our analysis, we build, in particular, multidirectional alias-free hexagonal wavelet bases and low-redundancy frames with optimal spatial decay. In addition, we show that a 2D cutting lemma can be used to subdivide the obtained wavelet systems in higher frequency rings so as to generate bases or frames that satisfy the ``parabolic scaling'' law enjoyed by curvelets and shearlets. Numerical experiments on high bit-rate image compression are conducted to illustrate the potential of the proposed systems.

math.NA↗

arXiv2019

Expression of Fractals Through Neural Network Functions

To help understand the underlying mechanisms of neural networks (NNs), several groups have, in recent years, studied the number of linear regions $\ell$ of piecewise linear functions generated by deep neural networks (DNN). In particular, they showed that $\ell$ can grow exponentially with the number of network parameters $p$, a property often used to explain the advantages of DNNs over shallow NNs in approximating complicated functions. Nonetheless, a simple dimension argument shows that DNNs cannot generate all piecewise linear functions with $\ell$ linear regions as soon as $\ell > p$. It is thus natural to seek to characterize specific families of functions with $\ell$ linear regions that can be constructed by DNNs. Iterated Function Systems (IFS) generate sequences of piecewise linear functions $F_k$ with a number of linear regions exponential in $k$. We show that, under mild assumptions, $F_k$ can be generated by a NN using only $\mathcal{O}(k)$ parameters. IFS are used extensively to generate, at low computational cost, natural-looking landscape textures in artificial images. They have also been proposed for compression of natural images, albeit with less commercial success. The surprisingly good performance of this fractal-based compression suggests that our visual system may lock in, to some extent, on self-similarities in images. The combination of this phenomenon with the capacity, demonstrated here, of DNNs to efficiently approximate IFS may contribute to the success of DNNs, particularly striking for image processing tasks, as well as suggest new algorithms for representing self similarities in images based on the DNN mechanism.

cs.LG↗

arXiv2019

ariaDNE: A Robustly Implemented Algorithm for Dirichlet Energy of the Normal

Point 1: Shape characterizers are metrics that quantify aspects of the overall geometry of a 3D digital surface. When computed for biological objects, the values of a shape characterizer are largely independent of homology interpretations and often contain a strong ecological and functional signal. Thus shape characterizers are useful for understanding evolutionary processes. Dirichlet Normal Energy (DNE) is a widely used shape characterizer in morphological studies. Point 2: Recent studies found that DNE is sensitive to various procedures for preparing 3D mesh from raw scan data, raising concerns regarding comparability and objectivity when utilizing DNE in morphological research. We provide a robustly implemented algorithm for computing the Dirichlet energy of the normal (ariaDNE) on 3D meshes. Point 3: We show through simulation that the effects of preparation-related mesh surface attributes such as triangle count, mesh representation, noise, smoothing and boundary triangles are much more limited on ariaDNE than DNE. Furthermore, ariaDNE retains the potential of DNE for biological studies, illustrated by its effectiveness in differentiating species by dietary preferences. Point 4: Use of ariaDNE can dramatically enhance assessment of ecological aspects of morphological variation by its stability under different 3D model acquisition methods and preparation procedure. Towards this goal, we provide scripts for computing ariaDNE and ariaDNE values for specimens used in previously published DNE analyses.

q-bio.QM↗

arXiv2019

Gaussian Process Landmarking on Manifolds

As a means of improving analysis of biological shapes, we propose an algorithm for sampling a Riemannian manifold by sequentially selecting points with maximum uncertainty under a Gaussian process model. This greedy strategy is known to be near-optimal in the experimental design literature, and appears to outperform the use of user-placed landmarks in representing the geometry of biological objects in our application. In the noiseless regime, we establish an upper bound for the mean squared prediction error (MSPE) in terms of the number of samples and geometric quantities of the manifold, demonstrating that the MSPE for our proposed sequential design decays at a rate comparable to the oracle rate achievable by any sequential or non-sequential optimal design; to our knowledge this is the first result of this type for sequential experimental design. The key is to link the greedy algorithm to reduced basis methods in the context of model reduction for partial differential equations. We expect this approach will find additional applications in other fields of research.

stat.ME↗

arXiv2019

Gaussian Process Landmarking for Three-Dimensional Geometric Morphometrics

We demonstrate applications of the Gaussian process-based landmarking algorithm proposed in [T. Gao, S.Z. Kovalsky, and I. Daubechies, SIAM Journal on Mathematics of Data Science (2019)] to geometric morphometrics, a branch of evolutionary biology centered at the analysis and comparisons of anatomical shapes, and compares the automatically sampled landmarks with the "ground truth" landmarks manually placed by evolutionary anthropologists; the results suggest that Gaussian process landmarks perform equally well or better, in terms of both spatial coverage and downstream statistical analysis. We provide a detailed exposition of numerical procedures and feature filtering algorithms for computing high-quality and semantically meaningful diffeomorphisms between disk-type anatomical surfaces.

stat.AP↗

arXiv2018

Stop memorizing: A data-dependent regularization framework for intrinsic pattern learning

Deep neural networks (DNNs) typically have enough capacity to fit random data by brute force even when conventional data-dependent regularizations focusing on the geometry of the features are imposed. We find out that the reason for this is the inconsistency between the enforced geometry and the standard softmax cross entropy loss. To resolve this, we propose a new framework for data-dependent DNN regularization, the Geometrically-Regularized-Self-Validating neural Networks (GRSVNet). During training, the geometry enforced on one batch of features is simultaneously validated on a separate batch using a validation loss consistent with the geometry. We study a particular case of GRSVNet, the Orthogonal-Low-rank Embedding (OLE)-GRSVNet, which is capable of producing highly discriminative features residing in orthogonal low-rank subspaces. Numerical experiments show that OLE-GRSVNet outperforms DNNs with conventional regularization when trained on real data. More importantly, unlike conventional DNNs, OLE-GRSVNet refuses to memorize random data or random labels, suggesting it only learns intrinsic patterns by reducing the memorizing capacity of the baseline DNN.

cs.CV↗