arXiv subjects

Marcus Carlsson

Publications and source records attributed to Marcus Carlsson.

32 records · Page 2Linked to original sources

arXiv2017

A Non-Convex Relaxation for Fixed-Rank Approximation

This paper considers the problem of finding a low rank matrix from observations of linear combinations of its elements. It is well known that if the problem fulfills a restricted isometry property (RIP), convex relaxations using the nuclear norm typically work well and come with theoretical performance guarantees. On the other hand these formulations suffer from a shrinking bias that can severely degrade the solution in the presence of noise. In this theoretical paper we study an alternative non-convex relaxation that in contrast to the nuclear norm does not penalize the leading singular values and thereby avoids this bias. We show that despite its non-convexity the proposed formulation will in many cases have a single local minimizer if a RIP holds. Our numerical tests show that our approach typically converges to a better solution than nuclear norm based alternatives even in cases when the RIP does not hold.

math.OC↗

arXiv2017

Nehari's theorem for convex domain Hankel and Toeplitz operators in several variables

We prove Nehari's theorem for integral Hankel and Toeplitz operators on simple convex polytopes in several variables. A special case of the theorem, generalizing the boundedness criterion of the Hankel and Toeplitz operators on the Paley-Wiener space, reads as follows. Let $Ξ= (0,1)^d$ be a $d$-dimensional cube, and for a distribution $f$ on $2Ξ$, consider the Hankel operator $$Γ_f (g)(x)=\int_Ξ f(x+y) g(y) \, dy, \quad x \inΞ.$$ Then $Γ_f$ extends to a bounded operator on $L^2(Ξ)$ if and only if there is a bounded function $b$ on $\mathbb{R}^d$ whose Fourier transform coincides with $f$ on $2Ξ$. This special case has an immediate application in matrix extension theory: every finite multi-level block Toeplitz matrix can be boundedly extended to an infinite multi-level block Toeplitz matrix. In particular, block Toeplitz operators with blocks which are themselves Toeplitz, can be extended to bounded infinite block Toeplitz operators with Toeplitz blocks.

math.FA↗

arXiv2017

Non-Convex Rank/Sparsity Regularization and Local Minima

This paper considers the problem of recovering either a low rank matrix or a sparse vector from observations of linear combinations of the vector or matrix elements. Recent methods replace the non-convex regularization with $\ell_1$ or nuclear norm relaxations. It is well known that this approach can be guaranteed to recover a near optimal solutions if a so called restricted isometry property (RIP) holds. On the other hand it is also known to perform soft thresholding which results in a shrinking bias which can degrade the solution. In this paper we study an alternative non-convex regularization term. This formulation does not penalize elements that are larger than a certain threshold making it much less prone to small solutions. Our main theoretical results show that if a RIP holds then the stationary points are often well separated, in the sense that their differences must be of high cardinality/rank. Thus, with a suitable initial solution the approach is unlikely to fall into a bad local minima. Our numerical tests show that the approach is likely to converge to a better solution than standard $\ell_1$/nuclear-norm relaxation even when starting from trivial initializations. In many cases our results can also be used to verify global optimality of our method.

math.OC↗

arXiv2016

Convergence of dual ascent in non-convex/non-differentiable optimization

We revisit the classical dual ascent algorithm for minimization of convex functionals in the presence of linear constraints, and give convergence results which apply even for non-convex functionals. We describe limit points in terms of the convex envelope. We also introduce a new augmented version, which is shown to have superior convergence properties, and provide new results even for convex but non-differentiable objective functionals (as well as non-convex). The results are applied to low rank approximation of a given matrix, subject to linear constraints. In particular, letting the linear constraints enforce Hankel structure of the respective matrices, the algorithms can be applied to complex frequency estimation. We provide numerical tests in this setting.

math.OC↗

arXiv2016

Convex envelopes for fixed rank approximation

A convex envelope for the problem of finding the best approximation to a given matrix with a prescribed rank is constructed. This convex envelope allows the usage of traditional optimization techniques when additional constraints are added to the finite rank approximation problem. Expression for the dependence of the convex envelope on the singular values of the given matrix is derived and global minimization properties are derived. The corresponding proximity operator is also studied.

math.FA↗

arXiv2016

Fast hyperbolic Radon transform represented as convolutions in log-polar coordinates

The hyperbolic Radon transform is a commonly used tool in seismic processing, for instance in seismic velocity analysis, data interpolation and for multiple removal. A direct implementation by summation of traces with different moveouts is computationally expensive for large data sets. In this paper we present a new method for fast computation of the hyperbolic Radon transforms. It is based on using a log-polar sampling with which the main computational parts reduce to computing convolutions. This allows for fast implementations by means of FFT. In addition to the FFT operations, interpolation procedures are required for switching between coordinates in the time-offset; Radon; and log-polar domains. Graphical Processor Units (GPUs) are suitable to use as a computational platform for this purpose, due to the hardware supported interpolation routines as well as optimized routines for FFT. Performance tests show large speed-ups of the proposed algorithm. Hence, it is suitable to use in iterative methods, and we provide examples for data interpolation and multiple removal using this approach.

math.NA↗

arXiv2016

Fixed-point algorithms for frequency estimation and structured low rank approximation

We develop fixed-point algorithms for the approximation of structured matrices with rank penalties. In particular we use these fixed-point algorithms for making approximations by sums of exponentials, or frequency estimation. For the basic formulation of the fixed-point algorithm we show that it converges to the minimum of the convex envelope of the original objective function along with its structured matrix constraint. It often happens that this solution agrees with the solution to the original minimization problem, and we provide a simple criterium for when this is true. We also provide more general fixed-point algorithms that can be used to treat the problems of making weighted approximations by sums of exponentials given equally or unequally spaced sampling. We apply the method to the case of missing data, although optimal convergence of the fixed-point algorithm is not guaranteed in this case. However, it turns out that the method often gives perfect reconstruction (up to machine precision) in such cases. We also discuss multidimensional extensions, and illustrate how the proposed algorithms can be used to recover sums of exponentials in several variables, but when samples are available only along a curve.

math.NA↗

arXiv2015

On the Kronecker and Caratheodory-Fejer theorems in several variables

Multivariate versions of the Kronecker theorem in the continuous multivariate setting has recently been published. These theorems characterize the symbols that give rise to finite rank multidimensional Hankel and Toeplitz type operators defined on general domains. In this paper we study how the additional assumption of positive semidefinite affects the characterization of the corresponding symbols, which we refer to as Carathéodory-Fejer type theorems. We show that these theorems become particularly transparent in the continuous setting, by providing elegant if and only if statements connecting the rank with sums of exponential functions. We also discuss how these objects can be discretized, giving rise to an interesting class of structured matrices that inherit these desirable properties from their continuous analogs. We describe how the continuous Kronecker theorem also applies to these structured matrices, given sufficient sampling. We also provide a new proof for the Carathéodory-Fejer theorem for block Toeplitz matrices, based on tools from tensor algebra.

math.FA↗

arXiv2015

Fast algorithms and efficient GPU implementations for the Radon transform and the back-projection operator represented as convolution operators

The Radon transform and its adjoint, the back-projection operator, can both be expressed as convolutions in log-polar coordinates. Hence, fast algorithms for the application of the operators can be constructed by using FFT, if data is resampled at log-polar coordinates. Radon data is typically measured on an equally spaced grid in polar coordinates, and reconstructions are represented (as images) in Cartesian coordinates. Therefore, in addition to FFT, several steps of interpolation have to be conducted in order to apply the Radon transform and the back-projection operator by means of convolutions. Both the interpolation and the FFT operations can be efficiently implemented on Graphical Processor Units (GPUs). For the interpolation, it is possible to make use of the fact that linear interpolation is hard-wired on GPUs, meaning that it has the same computational cost as direct memory access. Cubic order interpolation schemes can be constructed by combining linear interpolation steps which provides important computation speedup. We provide details about how the Radon transform and the back-projection can be implemented efficiently as convolution operators on GPUs. For large data sizes, speedups of about 10 times are obtained in relation to the computational times of other software packages based on GPU implementations of the Radon transform and the back-projection operator. Moreover, speedups of more than a 1000 times are obtained against the CPU-implementations provided in the MATLAB image processing toolbox.

math.NA↗

arXiv2015

Operator-Lipschitz estimates for the singular value functional calculus

We consider a functional calculus for compact operators, acting on the singular values rather than the spectrum, which appears frequently in applied mathematics. Necessary and sufficient conditions for this singular value functional calculus to be Lipschitz-continuous with respect to the Hilbert-Schmidt norm are given. We also provide sharp constants.

math.FA↗

arXiv2014

On linear differential equations with infinitely many derivatives

Differential equations with infinitely many derivatives, sometimes also referred to as ``nonlocal'' differential equations, appear frequently in branches of modern physics such as string theory, gravitation and cosmology. The goal of this paper is to show how to properly interpret and solve such equations, with a special focus on a solution method based on the Borel transform. This method is a far-reaching generalization of previous approaches (N. Barnaby and N. Kamran, Dynamics with infinitely many derivatives: the initial value problem. {\em J. High Energy Physics} 2008 no. 02, Paper 008, 40 pp.; P. Górka, H. Prado and E.G. Reyes, Functional calculus via Laplace transform and equations with infinitely many derivatives. {\em Journal of Mathematical Physics} 51 (2010), 103512; P. Górka, H. Prado and E.G. Reyes, The initial value problem for ordinary equations with infinitely many derivatives. {\em Classical and Quantum Gravity} 29 (2012), 065017). In particular we reconsider generalized initial value problems and disprove various conjectures found in the modern literature. We illustrate various phenomena that can occur with concrete examples, and we also treat efficient implementations of the theory.

math-ph↗

arXiv2013

Frequency estimation based on Hankel matrices and the alternating direction method of multipliers

We develop a parametric high-resolution method for the estimation of the frequency nodes of linear combinations of complex exponentials with exponential damping. We use Kronecker's theorem to formulate the associated nonlinear least squares problem as an optimization problem in the space of vectors generating Hankel matrices of fixed rank. Approximate solutions to this problem are obtained by using the alternating direction method of multipliers. Finally, we extract the frequency estimates from the con-eigenvectors of the solution Hankel matrix. The resulting algorithm is simple, easy to implement and can be applied to data with equally spaced samples with approximation weights, which for instance allows cases of missing data samples. By means of numerical simulations, we analyze and illustrate the excellent performance of the method, attaining the Cramér-Rao bound.

math.NA↗

arXiv2011

A fast alternating projection method for complex frequency estimation

The problem of approximating a sampled function using sums of a fixed number of complex exponentials is considered. We use alternating projections between fixed rank matrices and Hankel matrices to obtain such an approximation. Convergence, convergence rates and error estimates for this technique are proven, and fast algorithms are developed. We compare the numerical results obtain with the MUSIC and ESPRIT methods.

math.NA↗

arXiv2011

Alternating projections on non-tangential manifolds

We consider sequences $(B_k)_{k=0}^\infty$ of points obtained by projecting back and forth between two manifolds $\M_1$ and $\M_2$, and give conditions guaranteeing that the sequence converge to a limit $B_\infty\in\M_1\cap\M_2$. Our motivation is the study of algorithms based on finding the limit of such sequences, which have proven useful in a number of areas. The intersection is typically a set with desirable properties, but for which there is no efficient method of finding the closest point $B_{opt}$ in $\M_1\cap\M_2$. We prove not only that the sequence of alternating projections converges, but that the limit point is fairly close to $B_{opt}$, in a manner relative to the distance $\|B_0-B_{opt}\|$, thereby significantly improving earlier results in the field. A concrete example with applications to frequency estimation of signals is also presented.

math.NA↗