arXiv subjects

Housen Li

Publications and source records attributed to Housen Li.

28 records · Page 2Linked to original sources

arXiv2019

NETT: Solving Inverse Problems with Deep Neural Networks

Recovering a function or high-dimensional parameter vector from indirect measurements is a central task in various scientific areas. Several methods for solving such inverse problems are well developed and well understood. Recently, novel algorithms using deep learning and neural networks for inverse problems appeared. While still in their infancy, these techniques show astonishing performance for applications like low-dose CT or various sparse data problems. However, there are few theoretical results for deep learning in inverse problems. In this paper, we establish a complete convergence analysis for the proposed NETT (Network Tikhonov) approach to inverse problems. NETT considers data consistent solutions having small value of a regularizer defined by a trained neural network. We derive well-posedness results and quantitative error estimates, and propose a possible strategy for training the regularizer. Our theoretical results and framework are different from any previous work using neural networks for solving inverse problems. A possible data driven regularizer is proposed. Numerical results are presented for a tomographic sparse data problem, which demonstrate good performance of NETT even for unknowns of different type from the training data. To derive the convergence and convergence rates results we introduce a new framework based on the absolute Bregman distance generalizing the standard Bregman distance from the convex to the non-convex case.

math.NA↗

arXiv2019

The Essential Histogram

The histogram is widely used as a simple, exploratory display of data, but it is usually not clear how to choose the number and size of bins. We construct a confidence set of distribution functions that optimally address the two main tasks of the histogram: estimating probabilities and detecting features such as increases and modes in the distribution. We define the essential histogram as the histogram in the confidence set with the fewest bins. Thus the essential histogram is the simplest visualization of the data that optimally achieves the main tasks of the histogram. The only assumption we make is that the data are independent and identically distributed. We provide a fast algorithm for the essential histogram, and illustrate our methodology with examples. An R-package is available on CRAN.

math.ST↗

arXiv2019

Frame-constrained Total Variation Regularization for White Noise Regression

Despite the popularity and practical success of total variation (TV) regularization for function estimation, surprisingly little is known about its theoretical performance in a statistical setting. While TV regularization has been known for quite some time to be minimax optimal for denoising one-dimensional signals, for higher dimensions this remains elusive until today. In this paper we consider frame-constrained TV estimators including many well-known (overcomplete) frames in a white noise regression model, and prove their minimax optimality w.r.t. $L^q$-risk ($1\leq q<\infty$) up to a logarithmic factor in any dimension $d\geq 1$. Overcomplete frames are an established tool in mathematical imaging and signal recovery, and their combination with TV regularization has been shown to give excellent results in practice, which our theory now confirms. Our results rely on a novel connection between frame-constraints and certain Besov norms, and on an interpolation inequality to relate them to the risk functional.

math.ST↗

arXiv2019

Multiscale Change-point Segmentation: Beyond Step Functions

Modern multiscale type segmentation methods are known to detect multiple change-points with high statistical accuracy, while allowing for fast computation. Underpinning theory has been developed mainly for models that assume the signal as a piecewise constant function. In this paper this will be extended to certain function classes beyond such step functions in a nonparametric regression setting, revealing certain multiscale segmentation methods as robust to deviation from such piecewise constant functions. Our main finding is the adaptation over such function classes for a universal thresholding, which includes bounded variation functions, and (piecewise) Hölder functions of smoothness order $ 0 < α\le1$ as special cases. From this we derive statistical guarantees on feature detection in terms of jumps and modes. Another key finding is that these multiscale segmentation methods perform nearly (up to a log-factor) as well as the oracle piecewise constant segmentation estimator (with known jump locations), and the best piecewise constant approximants of the (unknown) true signal. Theoretical findings are examined by various numerical simulations.

math.ST↗

arXiv2018

Empirical Risk Minimization as Parameter Choice Rule for General Linear Regularization Methods

We consider the statistical inverse problem to recover $f$ from noisy measurements $Y = Tf + σξ$ where $ξ$ is Gaussian white noise and $T$ a compact operator between Hilbert spaces. Considering general reconstruction methods of the form $\hat f_α= q_α\left(T^*T\right)T^*Y$ with an ordered filter $q_α$, we investigate the choice of the regularization parameter $α$ by minimizing an unbiased estimate of the predictive risk $\mathbb E\left[\Vert Tf - T\hat f_α\Vert^2\right]$. The corresponding parameter $α_{\mathrm{pred}}$ and its usage are well-known in the literature, but oracle inequalities and optimality results in this general setting are unknown. We prove a (generalized) oracle inequality, which relates the direct risk $\mathbb E\left[\Vert f - \hat f_{α_{\mathrm{pred}}}\Vert^2\right]$ with the oracle prediction risk $\inf_{α>0}\mathbb E\left[\Vert Tf - T\hat f_α\Vert^2\right]$. From this oracle inequality we are then able to conclude that the investigated parameter choice rule is of optimal order. Finally we also present numerical simulations, which support the order optimality of the method and the quality of the parameter choice in finite sample situations.

math.NA↗

arXiv2018

A linear time algorithm for multiscale quantile simulation

Change-point problems have appeared in a great many applications for example cancer genetics, econometrics and climate change. Modern multiscale type segmentation methods are considered to be a statistically efficient approach for multiple change-point detection, which minimize the number of change-points under a multiscale side-constraint. The constraint threshold plays a critical role in balancing the data-fit and model complexity. However, the computation time of such a threshold is quadratic in terms of sample size $n$, making it impractical for large scale problems. In this paper we proposed an $\mathcal{O}(n)$ algorithm by utilizing the hidden quasiconvexity structure of the problem. It applies to all regression models in exponential family with arbitrary convex scale penalties. Simulations verify its computational efficiency and accuracy. An implementation is provided in R-package "linearQ" on CRAN.

stat.CO↗

arXiv2018

The Averaged Kaczmarz Iteration for Solving Inverse Problems

We introduce a new iterative regularization method for solving inverse problems that can be written as systems of linear or non-linear equations in Hilbert spaces. The proposed averaged Kaczmarz (AVEK) method can be seen as a hybrid method between the Landweber and the Kaczmarz method. As the Kaczmarz method, the proposed method only requires evaluation of one direct and one adjoint sub-problem per iterative update. On the other, similar to the Landweber iteration, it uses an average over previous auxiliary iterates which increases stability. We present a convergence analysis of the AVEK iteration. Further, detailed numerical studies are presented for a tomographic image reconstruction problem, namely the limited data problem in photoacoustic tomography. Thereby, the AVEK is compared with other iterative regularization methods including standard Landweber and Kaczmarz iterations, as well as recently proposed accelerated versions based on error minimizing relaxation strategies.

math.NA↗

arXiv2015

Variational Multiscale Nonparametric Regression: Smooth Functions

For the problem of nonparametric regression of smooth functions, we reconsider and analyze a constrained variational approach, which we call the MultIscale Nemirovski-Dantzig (MIND) estimator. This can be viewed as a multiscale extension of the Dantzig selector (\emph{Ann. Statist.}, 35(6): 2313--51, 2009) based on early ideas of Nemirovski (\emph{J. Comput. System Sci.}, 23:1--11, 1986). MIND minimizes a homogeneous Sobolev norm under the constraint that the multiresolution norm of the residual is bounded by a universal threshold. The main contribution of this paper is the derivation of convergence rates of MIND with respect to $L^q$-loss, $1 \le q \le \infty$, both almost surely and in expectation. To this end, we introduce the method of approximate source conditions. For a one-dimensional signal, these can be translated into approximation properties of $B$-splines. A remarkable consequence is that MIND attains almost minimax optimal rates simultaneously for a large range of Sobolev and Besov classes, which provides certain adaptation. Complimentary to the asymptotic analysis, we examine the finite sample performance of MIND by numerical simulations.

math.ST↗

arXiv2015

FDR-Control in Multiscale Change-point Segmentation

Fast multiple change-point segmentation methods, which additionally provide faithful statistical statements on the number, locations and sizes of the segments, have recently received great attention. In this paper, we propose a multiscale segmentation method, FDRSeg, which controls the false discovery rate (FDR) in the sense that the number of false jumps is bounded linearly by the number of true jumps. In this way, it adapts the detection power to the number of true jumps. We prove a non-asymptotic upper bound for its FDR in a Gaussian setting, which allows to calibrate the only parameter of FDRSeg properly. Change-point locations, as well as the signal, are shown to be estimated in a uniform sense at optimal minimax convergence rates up to a log-factor. The latter is w.r.t. $L^p$-risk, $p \ge 1$, over classes of step functions with bounded jump sizes and either bounded, or possibly increasing, number of change-points. FDRSeg can be efficiently computed by an accelerated dynamic program; its computational complexity is shown to be linear in the number of observations when there are many change-points. The performance of the proposed method is examined by comparisons with some state of the art methods on both simulated and real datasets. An R-package is available online.

math.ST↗

arXiv2013

Aggregated motion estimation for real-time MRI reconstruction

Real-time magnetic resonance imaging (MRI) methods generally shorten the measuring time by acquiring less data than needed according to the sampling theorem. In order to obtain a proper image from such undersampled data, the reconstruction is commonly defined as the solution of an inverse problem, which is regularized by a priori assumptions about the object. While practical realizations have hitherto been surprisingly successful, strong assumptions about the continuity of image features may affect the temporal fidelity of the estimated images. Here we propose a novel approach for the reconstruction of serial real-time MRI data which integrates the deformations between nearby frames into the data consistency term. The method is not required to be affine or rigid and does not need additional measurements. Moreover, it handles multi-channel MRI data by simultaneously determining the image and its coil sensitivity profiles in a nonlinear formulation which also adapts to non-Cartesian (e.g., radial) sampling schemes. Experimental results of a motion phantom with controlled speed and in vivo measurements of rapid tongue movements demonstrate image improvements in preserving temporal fidelity and removing residual artifacts.

math.NA↗