arXiv subjects

Johan Karlsson

Publications and source records attributed to Johan Karlsson.

54 records · Page 3Linked to original sources

arXiv2019

Multi-Marginal Optimal Mass Transport with Partial Information

During recent decades, there has been a substantial development in optimal mass transport theory and methods. In this work, we consider multi-marginal problems wherein only partial information of each marginal is available, which is a setup common in many inverse problems in, e.g., imaging and spectral estimation. By considering an entropy regularized approximation of the original transport problem, we propose an algorithm corresponding to a block-coordinate ascent of the dual problem, where Newton's algorithm is used to solve the sub-problems. In order to make this computationally tractable for large-scale settings, we utilize the tensor structure that arises in practical problems, allowing for computing projections of the multi-marginal transport plan using only matrix-vector operations of relatively small matrices. As illustrating examples, we apply the resulting method to tracking and barycenter problems in spatial spectral estimation. In particular, we show that the optimal mass transport framework allows for fusing information from different time steps, as well as from different sensor arrays, also when the sensor arrays are not jointly calibrated. Furthermore, we show that by incorporating knowledge of underlying dynamics in tracking scenarios, one may arrive at accurate spectral estimates, as well as faithful reconstructions of spectra corresponding to unobserved time points.

eess.SP↗

arXiv2019

Automatic Target Recognition Using Discrimination Based on Optimal Transport

The use of distances based on optimal transportation has recently shown promise for discrimination of power spectra. In particular, spectral estimation methods based on l1 regularization as well as covariance based methods can be shown to be robust with respect to such distances. These transportation distances provide a geometric framework where geodesics corresponds to smooth transition of spectral mass, and have been useful for tracking. In this paper, we investigate the use of these distances for automatic target recognition. We study the use of the Monge-Kantorovich distance compared to the standard l2 distance for classifying civilian vehicles based on SAR images. We use a version of the Monge-Kantorovich distance that applies also for the case where the spectra may have different total mass, and we formulate the optimization problem as a minimum flow problem that can be computed using efficient algorithms.

cs.CV↗

arXiv2019

Adapting The Secretary Hiring Problem for Optimal Hot-Cold Tier Placement under Top-$K$ Workloads

Top-K queries are an established heuristic in information retrieval. This paper presents an approach for optimal tiered storage allocation under stream processing workloads using this heuristic: those requiring the analysis of only the top-$K$ ranked most relevant, or most interesting, documents from a fixed-length stream, stream window, or batch job. In this workflow, documents are analyzed relevance with a user-specified interestingness function, on which they are ranked, the top-$K$ being selected (and hence stored) for further processing. This workflow allows human in the loop systems, including supervised machine learning, to prioritize documents. This scenario bears similarity to the classic Secretary Hiring Problem (SHP), and the expected rate of document writes, and document lifetime, can be modelled as a function of document index. We present parameter-based algorithms for storage tier placement, minimizing document storage and transport costs. We show that optimal parameter values are a function of these costs. It is possible to model application IO behaviour a priori for this class of workloads. When combined with tiered storage, the tractability of the probabilistic model of IO makes it easy to optimize tier allocation, yielding simple analytic solutions. This contrasts with (often complex) existing work on tiered storage optimization, which is either tightly coupled to specific use cases, or requires active monitoring of application IO load (a reactive approach) -- ill-suited to long-running or one-off operations common in the scientific computing domain.We validate our model with a trace-driven simulation of a bio-chemical model exploration, and give worked examples for two cloud storage case studies.

cs.DC↗

arXiv2018

Non-Coherent Sensor Fusion via Entropy Regularized Optimal Mass Transport

This work presents a method for information fusion in source localization applications. The method utilizes the concept of optimal mass transport in order to construct estimates of the spatial spectrum using a convex barycenter formulation. We introduce an entropy regularization term to the convex objective, which allows for low-complexity iterations of the solution algorithm and thus makes the proposed method applicable also to higher-dimensional problems. We illustrate the proposed method's inherent robustness to misalignment and miscalibration of the sensor arrays using numerical examples of localization in two dimensions.

eess.SP↗

arXiv2018

Lower bounds on the maximum delay margin by analytic interpolation

We study the delay margin problem in the context of recent works by T. Qi, J. Zhu, and J. Chen, where a sufficient condition for the maximal delay margin is formulated in terms of an interpolation problem obtained after introducing a rational approximation. Instead we omit the approximation step and solve the same problem directly using techniques from function theory and analytic interpolation. Furthermore, we introduce a constant shift in the domain of the interpolation problem. In this way we are able to improve on their lower bound for the maximum delay margin.

math.OC↗

arXiv2018

Data-driven nonsmooth optimization

In this work, we consider methods for solving large-scale optimization problems with a possibly nonsmooth objective function. The key idea is to first specify a class of optimization algorithms using a generic iterative scheme involving only linear operations and applications of proximal operators. This scheme contains many modern primal-dual first-order solvers like the Douglas-Rachford and hybrid gradient methods as special cases. Moreover, we show convergence to an optimal point for a new method which also belongs to this class. Next, we interpret the generic scheme as a neural network and use unsupervised training to learn the best set of parameters for a specific class of objective functions while imposing a fixed number of iterations. In contrast to other approaches of "learning to optimize", we present an approach which learns parameters only in the set of convergent schemes. As use cases, we consider optimization problems arising in tomographic reconstruction and image deconvolution, and in particular a family of total variation regularization problems.

math.OC↗

arXiv2018

State tracking of linear ensembles via optimal mass transport

We consider the problems of tracking an ensemble of indistinguishable agents with linear dynamics based only on output measurements. In this setting, the dynamics of the agents can be modeled by distribution flows in the state space and the measurements correspond to distributions in the output space. In this paper we formulate the corresponding state estimation problem using optimal mass transport theory with prior linear dynamics, and the optimal solution gives an estimate of the state trajectories of the ensemble. For general distributions of systems this can be formulated as a convex optimization problem which is computationally feasible with when the number of state dimensions is low. In the case where the marginal distributions are Gaussian, the problem is reformulated as a semidefinite programming and can be efficiently solved for tracking systems with a large number of states.

eess.SY↗

arXiv2018

Multidimensional Rational Covariance Extension with Approximate Covariance Matching

In our companion paper "Multidimensional rational covariance extension with applications to spectral estimation and image compression" we discussed the multidimensional rational covariance extension problem (RCEP), which has important applications in image processing, and spectral estimation in radar, sonar, and medical imaging. This is an inverse problem where a power spectrum with a rational absolutely continuous part is reconstructed from a finite set of moments. However, in most applications these moments are determined from observed data and are therefore only approximate, and RCEP may not have a solution. In this paper we extend the results to handle approximate covariance matching. We consider two problems, one with a soft constraint and the other one with a hard constraint, and show that they are connected via a homeomorphism. We also demonstrate that the problems are well-posed and illustrate the theory by examples in spectral estimation and texture generation.

math.OC↗

arXiv2017

Learning to solve inverse problems using Wasserstein loss

We propose using the Wasserstein loss for training in inverse problems. In particular, we consider a learned primal-dual reconstruction scheme for ill-posed inverse problems using the Wasserstein distance as loss function in the learning. This is motivated by miss-alignments in training data, which when using standard mean squared error loss could severely degrade reconstruction quality. We prove that training with the Wasserstein loss gives a reconstruction operator that correctly compensates for miss-alignments in certain cases, whereas training with the mean squared error gives a smeared reconstruction. Moreover, we demonstrate these effects by training a reconstruction algorithm using both mean squared error and optimal transport loss for a problem in computerized tomography.

cs.CV↗

arXiv2017

Generalized Sinkhorn iterations for regularizing inverse problems using optimal mass transport

The optimal mass transport problem gives a geometric framework for optimal allocation, and has recently gained significant interest in application areas such as signal processing, image processing, and computer vision. Even though it can be formulated as a linear programming problem, it is in many cases intractable for large problems due to the vast number of variables. A recent development to address this builds on an approximation with an entropic barrier term and solves the resulting optimization problem using Sinkhorn iterations. In this work we extend this methodology to a class of inverse problems. In particular we show that Sinkhorn-type iterations can be used to compute the proximal operator of the transport problem for large problems. A splitting framework is then used to solve inverse problems where the optimal mass transport cost is used for incorporating a priori information. We illustrate the method on problems in computerized tomography. In particular we consider a limited-angle computerized tomography problem, where a priori information is used to compensate for missing measurements.

math.OC↗

arXiv2016

Sylvester-based preconditioning for the waveguide eigenvalue problem

We consider a nonlinear eigenvalue problem (NEP) arising from absorbing boundary conditions in the study of a partial differential equation (PDE) describing a waveguide. We propose a new computational approach for this large scale NEP based on residual inverse iteration (Resinv) with preconditioned iterative solves. Similar to many preconditioned iterative methods for discretized PDEs, this approach requires the construction of an accurate and efficient preconditioner. For the waveguide eigenvalue problem, the associated linear system can be formulated as a generalized Sylvester equation. The equation is approximated by a low-rank correction of a Sylvester equation, which we use as a preconditioner. The action of the preconditioner is efficiently computed using the matrix equation version of the Sherman-Morrison-Woodbury (SMW) formula. We show how the preconditioner can be integrated into Resinv. The results are illustrated by applying the method to large-scale problems.

math.NA↗

arXiv2016

Multidimensional Rational Covariance Extension with Applications to Spectral Estimation and Image Compression

The rational covariance extension problem (RCEP) is an important problem in systems and control occurring in such diverse fields as control, estimation, system identification, and signal and image processing, leading to many fundamental theoretical questions. In fact, this inverse problem is a key component in many identification and signal processing techniques and plays a fundamental role in prediction, analysis, and modeling of systems and signals. It is well-known that the RCEP can be reformulated as a (truncated) trigonometric moment problem subject to a rationality condition. In this paper we consider the more general multidimensional trigonometric moment problem with a similar rationality constraint. This generalization creates many interesting new mathematical questions and also provides new insights into the original one-dimensional problem. A key concept in this approach is the complete smooth parametrization of all solutions, allowing solutions to be tuned to satisfy additional design specifications without violating the complexity constraints. As an illustration of the potential of this approach we apply our results to multidimensional spectral estimation and image compression. This is just a first step in this direction, and we expect that more elaborate tuning strategies will enhance our procedures in the future.

math.OC↗

arXiv2015

Time Localization and Capacity of Faster-Than-Nyquist Signaling

In this paper, we consider communication over the bandwidth limited analog white Gaussian noise channel using non-orthogonal pulses. In particular, we consider non-orthogonal transmission by signaling samples at a rate higher than the Nyquist rate. Using the faster-than-Nyquist (FTN) framework, Mazo showed that one may transmit symbols carried by sinc pulses at a higher rate than that dictated by Nyquist without loosing bit error rate. However, as we will show in this paper, such pulses are not necessarily well localized in time. In fact, assuming that signals in the FTN framework are well localized in time, one can construct a signaling scheme that violates the Shannon capacity bound. We also show directly that FTN signals are in general not well localized in time. Therefore, the results of Mazo do not imply that one can transmit more data per time unit without degrading performance in terms of error probability. We also consider FTN signaling in the case of pulses that are different from the sinc pulses. We show that one can use a precoding scheme of low complexity to remove the inter-symbol interference. This leads to the possibility of increasing the number of transmitted samples per time unit and compensate for spectral inefficiency due to signaling at the Nyquist rate of the non sinc pulses. We demonstrate the power of the precoding scheme by simulations.

cs.IT↗

arXiv2015

Robust Optimal Power Distribution for Hyperthermia Cancer Treatment

We consider an optimization problem for spatial power distribution generated by an array of transmitting elements. Using ultrasound hyperthermia cancer treatment as a motivating example, the signal design problem consists of optimizing the power distribution across the tumor and healthy tissue regions, respectively. The models used in the optimization problem are, however, invariably subject to errors. deposition as well as inefficient treatment. To combat such unknown model errors, we formulate a robust signal design framework that can take the uncertainty into account using a worst-case approach. This leads to a semi-infinite programming (SIP) robust design problem which we reformulate as a tractable convex problem, potentially has a wider range of applications.

cs.IT↗

arXiv2015

The role of the time-arrow in mean-square estimation of stochastic processes

The purpose of this paper is to explain a certain dichotomy between the information that the past and future values of a multivariate stochastic process carry about the present. More specifically, vector-valued, second-order stochastic processes may be deterministic in one time-direction and not the other. This phenomenon, which is absent in scalar-valued processes, is deeply rooted in the geometry of the shift-operator. The exposition and the examples we discuss are based on the work of Douglas, Shapiro and Shields on cyclic vectors of the backward shift and relate to classical ideas going back to Wiener and Kolmogorov. We focus on rank-one stochastic processes for which we present a characterization of all regular processes that are deterministic in the reverse time-direction. The paper builds on examples and the goal is to provide pertinent insights to a control engineering audience.

math.OC↗

arXiv2015

The Multidimensional Moment Problem with Complexity Constraint

A long series of previous papers have been devoted to the (one-dimensional) moment problem with nonnegative rational measure. The rationality assumption is a complexity constraint motivated by applications where a parameterization of the solution set in terms of a bounded finite number of parameters is required. In this paper we provide a complete solution of the multidimensional moment problem with a complexity constraint also allowing for solutions that require a singular measure added to the rational, absolutely continuous one. Such solutions occur on the boundary of a certain convex cone of solutions. In this paper we provide complete parameterizations of all such solutions. We also provide errata for a previous paper in this journal coauthored by one of the authors of the present paper.

math.OC↗

arXiv2012

Uncertainty Bounds for Spectral Estimation

The purpose of this paper is to study metrics suitable for assessing uncertainty of power spectra when these are based on finite second-order statistics. The family of power spectra which is consistent with a given range of values for the estimated statistics represents the uncertainty set about the "true" power spectrum. Our aim is to quantify the size of this uncertainty set using suitable notions of distance, and in particular, to compute the diameter of the set since this represents an upper bound on the distance between any choice of a nominal element in the set and the "true" power spectrum. Since the uncertainty set may contain power spectra with lines and discontinuities, it is natural to quantify distances in the weak topology---the topology defined by continuity of moments. We provide examples of such weakly-continuous metrics and focus on particular metrics for which we can explicitly quantify spectral uncertainty. We then consider certain high resolution techniques which utilize filter-banks for pre-processing, and compute worst-case a priori uncertainty bounds solely on the basis of the filter dynamics. This allows the a priori tuning of the filter-banks for improved resolution over selected frequency bands.

eess.SY↗

arXiv2006

An O(1) Solution to the Prefix Sum Problem on a Specialized Memory Architecture

In this paper we study the Prefix Sum problem introduced by Fredman. We show that it is possible to perform both update and retrieval in O(1) time simultaneously under a memory model in which individual bits may be shared by several words. We also show that two variants (generalizations) of the problem can be solved optimally in $Θ(\lg N)$ time under the comparison based model of computation.

cs.DS↗