arXiv subjects

Michael Unser

Publications and source records attributed to Michael Unser.

At least 91 records · Page 5Linked to original sources

arXiv2020

Duality Mapping for Schatten Matrix Norms

In this paper, we fully characterize the duality mapping over the space of matrices that are equipped with Schatten norms. Our approach is based on the analysis of the saturation of the Hölder inequality for Schatten norms. We prove in our main result that, for $p\in (1,\infty)$, the duality mapping over the space of real-valued matrices with Schatten-$p$ norm is a continuous and single-valued function and provide an explicit form for its computation. For the special case $p = 1$, the mapping is set-valued; by adding a rank constraint, we show that it can be reduced to a Borel-measurable single-valued function for which we also provide a closed-form expression.

math.FA↗

arXiv2020

A Note on BIBO Stability

The statements on the BIBO stability of continuous-time convolution systems found in engineering textbooks are often either too vague (because of lack of hypotheses) or mathematically incorrect. What is more troubling is that they usually exclude the identity operator. The purpose of this note is to clarify the issue while presenting some fixes. In particular, we show that a linear shift-invariant system is BIBO-stable in the $L_\infty$-sense if and only if its impulse response is included in the space of bounded Radon measures, which is a superset of $L_1(\mathbb{R})$ (Lebesgue's space of absolutely integrable functions). As we restrict the scope of this characterization to the convolution operators whose impulse response is a measurable function, we recover the classical statement.

math.FA↗

arXiv2020

Generating Sparse Stochastic Processes Using Matched Splines

We provide an algorithm to generate trajectories of sparse stochastic processes that are solutions of linear ordinary differential equations driven by Lévy white noises. A recent paper showed that these processes are limits in law of generalized compound-Poisson processes. Based on this result, we derive an off-the-grid algorithm that generates arbitrarily close approximations of the target process. Our method relies on a B-spline representation of generalized compound-Poisson processes. We illustrate numerically the validity of our approach.

math.ST↗

arXiv2020

Deep Neural Networks with Trainable Activations and Controlled Lipschitz Constant

We introduce a variational framework to learn the activation functions of deep neural networks. Our aim is to increase the capacity of the network while controlling an upper-bound of the actual Lipschitz constant of the input-output relation. To that end, we first establish a global bound for the Lipschitz constant of neural networks. Based on the obtained bound, we then formulate a variational problem for learning activation functions. Our variational problem is infinite-dimensional and is not computationally tractable. However, we prove that there always exists a solution that has continuous and piecewise-linear (linear-spline) activations. This reduces the original problem to a finite-dimensional minimization where an l1 penalty on the parameters of the activations favors the learning of sparse nonlinearities. We numerically compare our scheme with standard ReLU network and its variations, PReLU and LeakyReLU and we empirically demonstrate the practical aspects of our framework.

cs.LG↗

arXiv2020

A unifying representer theorem for inverse problems and machine learning

The standard approach for dealing with the ill-posedness of the training problem in machine learning and/or the reconstruction of a signal from a limited number of measurements is regularization. The method is applicable whenever the problem is formulated as an optimization task. The standard strategy consists in augmenting the original cost functional by an energy that penalizes solutions with undesirable behavior. The effect of regularization is very well understood when the penalty involves a Hilbertian norm. Another popular configuration is the use of an $\ell_1$-norm (or some variant thereof) that favors sparse solutions. In this paper, we propose a higher-level formulation of regularization within the context of Banach spaces. We present a general representer theorem that characterizes the solutions of a remarkably broad class of optimization problems. We then use our theorem to retrieve a number of known results in the literature---e.g., the celebrated representer theorem of machine leaning for RKHS, Tikhonov regularization, representer theorems for sparsity promoting functionals, the recovery of spikes---as well as a few new ones.

math.OC↗

arXiv2020

Joint Angular Refinement and Reconstruction for Single-Particle Cryo-EM

Single-particle cryo-electron microscopy (cryo-EM) reconstructs the three-dimensional (3D) structure of bio-molecules from a large set of 2D projection images with random and unknown orientations. A crucial step in the single-particle cryo-EM pipeline is 3D refinement, which resolves a high-resolution 3D structure from an initial approximate volume by refining the estimation of the orientation of each projection. In this work, we propose a new approach that refines the projection angles on the continuum. We formulate the optimization problem over the density map and the orientations jointly. The density map is updated using the efficient alternating-direction method of multipliers, while the orientations are updated through a semi-coordinate-wise gradient descent for which we provide an explicit derivation of the gradient. Our method eliminates the requirement for a fine discretization of the orientation space and does away with the classical but computationally expensive template-matching step. Numerical results demonstrate the feasibility and performance of our approach compared to several baselines.

eess.IV↗

arXiv2020

Fast Rotational Sparse Coding

We propose an algorithm for rotational sparse coding along with an efficient implementation using steerability. Sparse coding (also called dictionary learning) is an important technique in image processing, useful in inverse problems, compression, and analysis; however, the usual formulation fails to capture an important aspect of the structure of images: images are formed from building blocks, e.g., edges, lines, or points, that appear at different locations, orientations, and scales. The sparse coding problem can be reformulated to explicitly account for these transforms, at the cost of increased computation. In this work, we propose an algorithm for a rotational version of sparse coding that is based on K-SVD with additional rotation operations. We then propose a method to accelerate these rotations by learning the dictionary in a steerable basis. Our experiments on patch coding and texture classification demonstrate that the proposed algorithm is fast enough for practical use and compares favorably to standard sparse coding.

eess.IV↗

arXiv2020

Dictionary Learning for Two-Dimensional Kendall Shapes

We propose a novel sparse dictionary learning method for planar shapes in the sense of Kendall, namely configurations of landmarks in the plane considered up to similitudes. Our shape dictionary method provides a good trade-off between algorithmic simplicity and faithfulness with respect to the nonlinear geometric structure of Kendall's shape space. Remarkably, it boils down to a classical dictionary learning formulation modified using complex weights. Existing dictionary learning methods extended to nonlinear spaces either map the manifold to a reproducing kernel Hilbert space or to a tangent space. The first approach is unnecessarily heavy in the case of Kendall's shape space and causes the geometrical understanding of shapes to be lost, while the second one induces distortions and theoretical complexity. Our approach does not suffer from these drawbacks. Instead of embedding the shape space into a linear space, we rely on the hyperplane of centered configurations, including pre-shapes from which shapes are defined as rotation orbits. In this linear space, the dictionary atoms are scaled and rotated using complex weights before summation. Furthermore, our formulation is more general than Kendall's original one: it applies to discretely-defined configurations of landmarks as well as continuously-defined interpolating curves. We implemented our algorithm by adapting the method of optimal directions combined to a Cholesky-optimized order recursive matching pursuit. An interesting feature of our shape dictionary is that it produces visually realistic atoms, while guaranteeing reconstruction accuracy. Its efficiency can mostly be attributed to a clear formulation of the framework with complex numbers. We illustrate the strong potential of our approach for the characterization of datasets of shapes up to similitudes and the analysis of patterns in deforming 2D shapes.

eess.IV↗

arXiv2019

Three-Dimensional Optical Diffraction Tomography with Lippmann-Schwinger Model

A broad class of imaging modalities involve the resolution of an inverse-scattering problem. Among them, three-dimensional optical diffraction tomography (ODT) comes with its own challenges. These include a limited range of views, a large size of the sample with respect to the illumination wavelength, and optical aberrations that are inherent to the system itself. In this work, we present an accurate and efficient implementation of the forward model. It relies on the exact (nonlinear) Lippmann-Schwinger equation. We address several crucial issues such as the discretization of the Green function, the computation of the far field, and the estimation of the incident field. We then deploy this model in a regularized variational-reconstruction framework and show on both simulated and real data that it leads to substantially better reconstructions than the approximate models that are traditionally used in ODT.

eess.IV↗

arXiv2019

Method for Assessing the Fidelity of Optical Diffraction Tomography Reconstruction Methods

We use a spatial light modulator in a diffraction tomographic system to assess the accuracy of different refractive index reconstruction algorithms. Optical phase conjugation principles through complex media, allows us to quantify the error for different refractive index reconstruction algorithms without access to the ground truth. To our knowledge, this is the first assessment technique that uses structured illumination experimentally to test the accuracy of different reconstruction schemes.

physics.optics↗

arXiv2019

Native Banach spaces for splines and variational inverse problems

We propose a systematic construction of native Banach spaces for general spline-admissible operators ${\rm L}$. In short, the native space for ${\rm L}$ and the (dual) norm $\|\cdot\|_{\mathcal{X}'}$ is the largest space of functions $f: \mathbb{R}^d \to \mathbb{R}$ such that $\|{\rm L} f\|_{\mathcal{X}'}<\infty$, subject to the constraint that the growth-restricted null space of ${\rm L}$be finite-dimensional. This space, denoted by $\mathcal{X}'_{\rm L}$, is specified as the dual of the pre-native space $\mathcal{X}_{\rm L}$, which is itself obtained through a suitable completion process. The main difference with prior constructions (e.g., reproducing kernel Hilbert spaces) is that our approach involves test functions rather than sums of atoms (e.g, kernels), which makes it applicable to a much broader class of norms, including total variation. Under specific admissibility and compatibility hypotheses, we lay out the direct-sum topology of $\mathcal{X}_{\rm L}$ and $\mathcal{X}'_{\rm L}$, and identify the whole family of equivalent norms. Our construction ensures that the native space and its pre-dual are endowed with a fundamental Schwartz-Banach property. In practical terms, this means that $\mathcal{X}'_{\rm L}$ is rich enough to reproduce any function with an arbitrary degree of precision.

math.FA↗

arXiv2019

Support and Approximation Properties of Hermite Splines

In this paper, we formally investigate two mathematical aspects of Hermite splines which translate to features that are relevant to their practical applications. We first demonstrate that Hermite splines are maximally localized in the sense that their support sizes are minimal among pairs of functions with identical reproduction properties. Then, we precisely quantify the approximation power of Hermite splines for reconstructing functions and their derivatives, and show that they are asymptotically identical to cubic B-splines for these tasks. Hermite splines therefore combine optimal localization and excellent approximation power, while retaining interpolation properties and closed-form expression, in contrast to existing similar approaches. These findings shed a new light on the convenience of Hermite splines for use in computer graphics and geometrical design.

math.NA↗

arXiv2019

A representer theorem for deep neural networks

We propose to optimize the activation functions of a deep neural network by adding a corresponding functional regularization to the cost function. We justify the use of a second-order total-variation criterion. This allows us to derive a general representer theorem for deep neural networks that makes a direct connection with splines and sparsity. Specifically, we show that the optimal network configuration can be achieved with activation functions that are nonuniform linear splines with adaptive knots. The bottom line is that the action of each neuron is encoded by a spline whose parameters (including the number of knots) are optimized during the training procedure. The scheme results in a computational structure that is compatible with the existing deep-ReLU, parametric ReLU, APL (adaptive piecewise-linear) and MaxOut architectures. It also suggests novel optimization challenges, while making the link with $\ell_1$ minimization and sparsity-promoting techniques explicit.

stat.ML↗

arXiv2019

Self-Supervised Deep Active Accelerated MRI

We propose to simultaneously learn to sample and reconstruct magnetic resonance images (MRI) to maximize the reconstruction quality given a limited sample budget, in a self-supervised setup. Unlike existing deep methods that focus only on reconstructing given data, thus being passive, we go beyond the current state of the art by considering both the data acquisition and the reconstruction process within a single deep-learning framework. As our network learns to acquire data, the network is active in nature. In order to do so, we simultaneously train two neural networks, one dedicated to reconstruction and the other to progressive sampling, each with an automatically generated supervision signal that links them together. The two supervision signals are created through Monte Carlo tree search (MCTS). MCTS returns a better sampling pattern than what the current sampling network can give and, thus, a better final reconstruction. The sampling network is trained to mimic the MCTS results using the previous sampling network, thus being enhanced. The reconstruction network is trained to give the highest reconstruction quality, given the MCTS sampling pattern. Through this framework, we are able to train the two networks without providing any direct supervision on sampling.

cs.CV↗

arXiv2019

Pocket Guide to Solve Inverse Problems with GlobalBioIm

GlobalBioIm is an open-source MATLAB library for solving inverse problems. The library capitalizes on the strong commonalities between forward models to standardize the resolution of a wide range of imaging inverse problems. Endowed with an operator-algebra mechanism, GlobalBioIm allows one to easily solve inverse problems by combining elementary modules in a lego-like fashion. This user-friendly toolbox gives access to cutting-edge reconstruction algorithms, while its high modularity makes it easily extensible to new modalities and novel reconstruction methods. We expect GlobalBioIm to respond to the needs of imaging scientists looking for reliable and easy-to-use computational tools for solving their inverse problems. In this paper, we present in detail the structure and main features of the library. We also illustrate its flexibility with examples from multichannel deconvolution microscopy.

cs.MS↗

arXiv2018

Periodic Splines and Gaussian Processes for the Resolution of Linear Inverse Problems

This paper deals with the resolution of inverse problems in a periodic setting or, in other terms, the reconstruction of periodic continuous-domain signals from their noisy measurements. We focus on two reconstruction paradigms: variational and statistical. In the variational approach, the reconstructed signal is solution to an optimization problem that establishes a tradeoff between fidelity to the data and smoothness conditions via a quadratic regularization associated to a linear operator. In the statistical approach, the signal is modeled as a stationary random process defined from a Gaussian white noise and a whitening operator; one then looks for the optimal estimator in the mean-square sense. We give a generic form of the reconstructed signals for both approaches, allowing for a rigorous comparison of the two.We fully characterize the conditions under which the two formulations yield the same solution, which is a periodic spline in the case of sampling measurements. We also show that this equivalence between the two approaches remains valid on simulations for a broad class of problems. This extends the practical range of applicability of the variational method.

math.OC↗

arXiv2018

Direct Reconstruction of Saturated Samples in Band-Limited OFDM Signals

Given a set of samples, a few of them being possibly saturated, we propose an efficient algorithm in order to cancel saturation while reconstructing band-limited signals. Our method satisfies a minimum-loss constraint and relies on sinc-related bases. It involves matrix inversion and is a direct, non-iterative approach. It consists of two main steps: (i) regression, to estimate the expansion coefficients of the signal model; (ii) interpolation, to restore an estimated value for those samples that are saturated. Because the proposed method is free from tuning parameters, it is hardware-friendly and we expect that it will be particularly useful in the context of orthogonal frequency-division multiplexing. There, the high peak-to-average power ratio of the transmitted signal results in a challenging decoding stage in the presence of saturation, which causes significant decoding errors due to the nonlinearity of amplifiers and receivers, ultimately resulting in band distortion and information loss. Our experiments on realistic simulations confirm that our proposed reconstruction of the saturated samples can significantly reduce transmission errors in modern high-throughput digital-communication receivers.

eess.SP↗

arXiv2018

Diverse M-Best Solutions by Dynamic Programming

Many computer vision pipelines involve dynamic programming primitives such as finding a shortest path or the minimum energy solution in a tree-shaped probabilistic graphical model. In such cases, extracting not merely the best, but the set of M-best solutions is useful to generate a rich collection of candidate proposals that can be used in downstream processing. In this work, we show how M-best solutions of tree-shaped graphical models can be obtained by dynamic programming on a special graph with M layers. The proposed multi-layer concept is optimal for searching M-best solutions, and so flexible that it can also approximate M-best diverse solutions. We illustrate the usefulness with applications to object detection, panorama stitching and centerline extraction. Note: We have observed that an assumption in section 4 of our paper is not always fulfilled, see the attached corrigendum for details.

cs.CV↗