arXiv subjects

Guy Gilboa

Publications and source records attributed to Guy Gilboa.

51 records · Page 3Linked to original sources

arXiv2020

Revealing Stable and Unstable Modes of Generic Denoisers through Nonlinear Eigenvalue Analysis

In this paper, we propose to analyze stable and unstable modes of generic image denoisers through nonlinear eigenvalue analysis. We attempt to find input images for which the output of a black-box denoiser is proportional to the input. We treat this as a nonlinear eigenvalue problem. This has potentially wide implications, since most image processing algorithms can be viewed as generic nonlinear operators. We introduce a generalized nonlinear power-method to solve eigenproblems for such black-box operators. Using this method we reveal stable modes of nonlinear denoisers. These modes are optimal inputs for the denoiser, achieving superior PSNR in noise removal. Analogously to the linear case (low-pass-filter), such stable modes are eigenfunctions corresponding to large eigenvalues, characterized by large piece-wise-smooth structures. We also provide a method to generate the complementary, most unstable modes, which the denoiser suppresses strongly. These modes are textures with small eigenvalues. We validate the method using total-variation (TV) and demonstrate it on the EPLL denoiser (Zoran-Weiss). Finally, we suggest an encryption-decryption application.

math.NA↗

arXiv2020

Unsupervised Enhancement of Real-World Depth Images Using Tri-Cycle GAN

Low quality depth poses a considerable challenge to computer vision algorithms. In this work we aim to enhance highly degraded, real-world depth images acquired by a low-cost sensor, for which an analytical noise model is unavailable. In the absence of clean ground-truth, we approach the task as an unsupervised domain-translation between the low-quality sensor domain and a high-quality sensor domain, represented using two unpaired training sets. We employ the highly-successful Cycle-GAN to this task, but find it to perform poorly in this case. Identifying the sources of the failure, we introduce several modifications to the framework, including a larger generator architecture, depth-specific losses that take into account missing pixels, and a novel Tri-Cycle loss which promotes information-preservation while addressing the asymmetry between the domains. We show that the resulting framework dramatically improves over the original Cycle-GAN both visually and quantitatively, extending its applicability to more challenging and asymmetric translation tasks.

cs.CV↗

arXiv2019

Introducing the p-Laplacian Spectra

In this work we develop a nonlinear decomposition, associated with nonlinear eigenfunctions of the p-Laplacian for p \in (1, 2). With this decomposition we can process signals of different degrees of smoothness. We first analyze solutions of scale spaces, generated by γ-homogeneous operators, γ \in R. An analytic solution is formulated when the scale space is initialized with a nonlinear eigenfunction of the respective operator. We show that the flow is extinct in finite time for γ \in [0, 1). A main innovation in this study is concerned with operators of fractional homogeneity, which require the mathematical framework of fractional calculus. The proposed transform rigorously defines the notions of decomposition, reconstruction, filtering and spectrum. The theory is applied to the p-Laplacian operator, where the tools developed in this framework are demonstrated. Keywords: Nonlinear spectra, filtering, shape preserving flows, p-Laplacian, nonlinear eigenfunctions.

math.AP↗

arXiv2019

Optoacoustic Model-Based Inversion Using Anisotropic Adaptive Total-Variation Regularization

In optoacoustic tomography, image reconstruction is often performed with incomplete or noisy data, leading to reconstruction errors. Significant improvement in reconstruction accuracy may be achieved in such cases by using nonlinear regularization schemes, such as total-variation minimization and $L_1$-based sparsity-preserving schemes. In this paper, we introduce a new framework for optoacoustic image reconstruction based on adaptive anisotropic total-variation regularization, which is more capable of preserving complex boundaries than conventional total-variation regularization. The new scheme is demonstrated in numerical simulations on blood-vessel images \textcolor{black} {as well as on experimental data} and is shown to be more capable than the total-variation-$L_1$ scheme in enhancing image contrast.

eess.IV↗

arXiv2019

Image-Guided Depth Sampling and Reconstruction

Depth acquisition, based on active illumination, is essential for autonomous and robotic navigation. LiDARs (Light Detection And Ranging) with mechanical, fixed, sampling templates are commonly used in today's autonomous vehicles. An emerging technology, based on solid-state depth sensors, with no mechanical parts, allows fast, adaptive, programmable scans. In this paper, we investigate the topic of adaptive, image-driven, sampling and reconstruction strategies. First, we formulate a piece-wise linear depth model with several tolerance parameters and estimate its validity for indoor and outdoor scenes. Our model and experiments predict that, in the optimal case, about 20-60 piece-wise linear structures can approximate well a depth map. This translates to a depth-to-image sampling ratio of about 1/1200. We propose a simple, generic, sampling and reconstruction algorithm, based on super-pixels. We reach a sampling rate which is still far from the optimal case. However, our sampling improves grid and random sampling, consistently, for a wide variety of reconstruction methods. Moreover, our proposed reconstruction achieves state-of-the-art results, compared to image-guided depth completion algorithms, reducing the required sampling rate by a factor of 3-4. A single-pixel depth camera built in our lab illustrates the concept.

cs.CV↗

arXiv2018

Adaptive Anisotropic Total Variation - A Nonlinear Spectral Analysis

A fundamental concept in solving inverse problems is the use of regularizers, which yield more physical and less-oscillatory solutions. Total variation (TV) has been widely used as an edge-preserving regularizer. However, objects are often over-regularized by TV, becoming blob-like convex structures of low curvature. This phenomenon was explained mathematically in the analysis of Andreau et al. They have shown that a TV regularizer can spatially preserve perfectly sets which are nonlinear eigenfunctions of the form $λu \in \partial J_{TV}(u)$, where $\partial J_{TV}(u)$ is the TV subdifferential. For TV, these shapes are convex sets of low-curvature. A compelling approach to better preserve structures is to use anisotropic functionals, which adapt the regularization in an image-driven manner, with strong regularization along edges and low across them. This follows earlier ideas of Weickert on anisotropic diffusion, which do not stem directly from functional minimization. Adaptive anisotropic TV (A$^2$TV) was successfully used in several studies in the past decade. However, until now there is no theory formulating the type of structures which can be perfectly preserved. In this study we address this question. We rely on a recently developed theory of Burger et al on nonlinear spectral analysis of one-homogeneous functionals. We have that eigenfunction sets, admitting $λu \in \partial J_{A^2TV}(u)$, are perfectly preserved under A$^2$TV-flow or minimization with $L^2$ square fidelity. We thus investigate these eigenfunctions theoretically and numerically. We prove non-convex sets can be eigenfunctions in certain conditions and provide numerical results which characterize well the relations between the degree of local anisotropy of the functional and the admitted maximal curvature....

eess.IV↗

arXiv2017

Blind Facial Image Quality Enhancement using Non-Rigid Semantic Patches

We propose to combine semantic data and registration algorithms to solve various image processing problems such as denoising, super-resolution and color-correction. It is shown how such new techniques can achieve significant quality enhancement, both visually and quantitatively, in the case of facial image enhancement. Our model assumes prior high quality data of the person to be processed, but no knowledge of the degradation model. We try to overcome the classical processing limits by using semantically-aware patches, with adaptive size and location regions of coherent structure and context, as building blocks. The method is demonstrated on the problem of cellular photography enhancement of dark facial images for different identities, expressions and poses.

cs.CV↗

arXiv2017

Learning Filter Functions in Regularisers by Minimising Quotients

Learning approaches have recently become very popular in the field of inverse problems. A large variety of methods has been established in recent years, ranging from bi-level learning to high-dimensional machine learning techniques. Most learning approaches, however, only aim at fitting parametrised models to favourable training data whilst ignoring misfit training data completely. In this paper, we follow up on the idea of learning parametrised regularisation functions by quotient minimisation as established in [3]. We extend the model therein to include higher-dimensional filter functions to be learned and allow for fit- and misfit-training data consisting of multiple functions. We first present results resembling behaviour of well-established derivative-based sparse regularisers like total variation or higher-order total variation in one-dimension. Our second and main contribution is the introduction of novel families of non-derivative-based regularisers. This is accomplished by learning favourable scales and geometric properties while at the same time avoiding unfavourable ones.

math.OC↗

arXiv2017

Nonlinear Spectral Image Fusion

In this paper we demonstrate that the framework of nonlinear spectral decompositions based on total variation (TV) regularization is very well suited for image fusion as well as more general image manipulation tasks. The well-localized and edge-preserving spectral TV decomposition allows to select frequencies of a certain image to transfer particular features, such as wrinkles in a face, from one image to another. We illustrate the effectiveness of the proposed approach in several numerical experiments, including a comparison to the competing techniques of Poisson image editing, linear osmosis, wavelet fusion and Laplacian pyramid fusion. We conclude that the proposed spectral TV image decomposition framework is a valuable tool for semi- and fully-automatic image editing and fusion.

cs.CV↗

arXiv2016

Flows Generating Nonlinear Eigenfunctions

Nonlinear variational methods have become very powerful tools for many image processing tasks. Recently a new line of research has emerged, dealing with nonlinear eigenfunctions induced by convex functionals. This has provided new insights and better theoretical understanding of convex regularization and introduced new processing methods. However, the theory of nonlinear eigenvalue problems is still at its infancy. We present a new flow that can generate nonlinear eigenfunctions of the form $T(u)=λu$, where $T(u)$ is a nonlinear operator and $λ\in \mathbb{R} $ is the eigenvalue. We develop the theory where $T(u)$ is a subgradient element of a regularizing one-homogeneous functional, such as total-variation (TV) or total-generalized-variation (TGV). We introduce two flows: a forward flow and an inverse flow; for which the steady state solution is a nonlinear eigenfunction. The forward flow monotonically smooths the solution (with respect to the regularizer) and simultaneously increases the $L^2$ norm. The inverse flow has the opposite characteristics. For both flows, the steady state depends on the initial condition, thus different initial conditions yield different eigenfunctions. This enables a deeper investigation into the space of nonlinear eigenfunctions, allowing to produce numerically diverse examples, which may be unknown yet. In addition we suggest an indicator to measure the affinity of a function to an eigenfunction and relate it to pseudo-eigenfunctions in the linear case.

cs.CV↗

arXiv2016

Spectral Decompositions using One-Homogeneous Functionals

This paper discusses the use of absolutely one-homogeneous regularization functionals in a variational, scale space, and inverse scale space setting to define a nonlinear spectral decomposition of input data. We present several theoretical results that explain the relation between the different definitions. Additionally, results on the orthogonality of the decomposition, a Parseval-type identity and the notion of generalized (nonlinear) eigenvectors closely link our nonlinear multiscale decompositions to the well-known linear filtering theory. Numerical results are used to illustrate our findings.

math.NA↗

arXiv2015

Semi-Inner-Products for Convex Functionals and Their Use in Image Decomposition

Semi-inner-products in the sense of Lumer are extended to convex functionals. This yields a Hilbert-space like structure to convex functionals in Banach spaces. In particular, a general expression for semi-inner-products with respect to one homogeneous functionals is given. Thus one can use the new operator for the analysis of total variation and higher order functionals like total-generalized-variation (TGV). Having a semi-inner-product, an angle between functions can be defined in a straightforward manner. It is shown that in the one homogeneous case the Bregman distance can be expressed in terms of this newly defined angle. In addition, properties of the semi-inner-product of nonlinear eigenfunctions induced by the functional are derived. We use this construction to state a sufficient condition for a perfect decomposition of two signals and suggest numerical measures which indicate when those conditions are approximately met.

math.NA↗

arXiv2015

Separation Surfaces in the Spectral TV Domain for Texture Decomposition

In this paper we introduce a novel notion of separation surfaces for image decomposition. A surface is embedded in the spectral total-variation (TV) three dimensional domain and encodes a spatially-varying separation scale. The method allows good separation of textures with gradually varying pattern-size, pattern-contrast or illumination. The recently proposed total variation spectral framework is used to decompose the image into a continuum of textural scales. A desired texture, within a scale range, is found by fitting a surface to the local maximal responses in the spectral domain. A band above and below the surface, referred to as the \textit{Texture Stratum}, defines for each pixel the adaptive scale-range of the texture. Based on the decomposition an application is proposed which can attenuate or enhance textures in the image in a very natural and visually convincing manner.

cs.CV↗

arXiv2015

Nonlinear Spectral Analysis via One-homogeneous Functionals - Overview and Future Prospects

We present in this paper the motivation and theory of nonlinear spectral representations, based on convex regularizing functionals. Some comparisons and analogies are drawn to the fields of signal processing, harmonic analysis and sparse representations. The basic approach, main results and initial applications are shown. A discussion of open problems and future directions concludes this work.

math.SP↗

arXiv2015

Spectral Representations of One-Homogeneous Functionals

This paper discusses a generalization of spectral representations related to convex one-homogeneous regularization functionals, e.g. total variation or $\ell^1$-norms. Those functionals serve as a substitute for a Hilbert space structure (and the related norm) in classical linear spectral transforms, e.g. Fourier and wavelet analysis. We discuss three meaningful definitions of spectral representations by scale space and variational methods and prove that (nonlinear) eigenfunctions of the regularization functionals are indeed atoms in the spectral representation. Moreover, we verify further useful properties related to orthogonality of the decomposition and the Parseval identity. The spectral transform is motivated by total variation and further developed to higher order variants. Moreover, we show that the approach can recover Fourier analysis as a special case using an appropriate $\ell^1$-type functional and discuss a coupled sparsity example.

math.NA↗