arXiv subjects

Jian-Feng Cai

Publications and source records attributed to Jian-Feng Cai.

At least 55 records · Page 3Linked to original sources

arXiv2022

Provable Tensor-Train Format Tensor Completion by Riemannian Optimization

The tensor train (TT) format enjoys appealing advantages in handling structural high-order tensors. The recent decade has witnessed the wide applications of TT-format tensors from diverse disciplines, among which tensor completion has drawn considerable attention. Numerous fast algorithms, including the Riemannian gradient descent (RGrad), have been proposed for the TT-format tensor completion. However, the theoretical guarantees of these algorithms are largely missing or sub-optimal, partly due to the complicated and recursive algebraic operations in TT-format decomposition. Moreover, existing results established for the tensors of other formats, for example, Tucker and CP, are inapplicable because the algorithms treating TT-format tensors are substantially different and more involved. In this paper, we provide, to our best knowledge, the first theoretical guarantees of the convergence of RGrad algorithm for TT-format tensor completion, under a nearly optimal sample size condition. The RGrad algorithm converges linearly with a constant contraction rate that is free of tensor condition number without the necessity of re-conditioning. We also propose a novel approach, referred to as the sequential second-order moment method, to attain a warm initialization under a similar sample size requirement. As a byproduct, our result even significantly refines the prior investigation of RGrad algorithm for matrix completion. Lastly, statistically (near) optimal rate is derived for RGrad algorithm if the observed entries consist of random sub-Gaussian noise. Numerical experiments confirm our theoretical discovery and showcase the computational speedup gained by the TT-format decomposition.

cs.LG↗

arXiv2021

The global landscape of phase retrieval I: perturbed amplitude models

A fundamental task in phase retrieval is to recover an unknown signal $\vx\in \Rn$ from a set of magnitude-only measurements $y_i=\abs{\nj{\va_i,\vx}}, \; i=1,\ldots,m$. In this paper, we propose two novel perturbed amplitude models (PAMs) which have non-convex and quadratic-type loss function. When the measurements $ \va_i \in \Rn$ are Gaussian random vectors and the number of measurements $m\ge Cn$, we rigorously prove that the PAMs admit no spurious local minimizers with high probability, i.e., the target solution $ \vx$ is the unique global minimizer (up to a global phase) and the loss function has a negative directional curvature around each saddle point. Thanks to the well-tamed benign geometric landscape, one can employ the vanilla gradient descent method to locate the global minimizer $\vx$ (up to a global phase) without spectral initialization. We carry out extensive numerical experiments to show that the gradient descent algorithm with random initialization outperforms state-of-the-art algorithms with spectral initialization in empirical success rate and convergence speed.

math.NA↗

arXiv2021

The global landscape of phase retrieval II: quotient intensity models

A fundamental problem in phase retrieval is to reconstruct an unknown signal from a set of magnitude-only measurements. In this work we introduce three novel quotient intensity-based models (QIMs) based a deep modification of the traditional intensity-based models. A remarkable feature of the new loss functions is that the corresponding geometric landscape is benign under the optimal sampling complexity. When the measurements $ a_i\in \Rn$ are Gaussian random vectors and the number of measurements $m\ge Cn$, the QIMs admit no spurious local minimizers with high probability, i.e., the target solution $ x$ is the unique global minimizer (up to a global phase) and the loss function has a negative directional curvature around each saddle point. Such benign geometric landscape allows the gradient descent methods to find the global solution $x$ (up to a global phase) without spectral initialization.

math.NA↗

arXiv2021

Sparse Signal Recovery from Phaseless Measurements via Hard Thresholding Pursuit

In this paper, we consider the sparse phase retrieval problem, recovering an $s$-sparse signal $\bm{x}^{\natural}\in\mathbb{R}^n$ from $m$ phaseless samples $y_i=|\langle\bm{x}^{\natural},\bm{a}_i\rangle|$ for $i=1,\ldots,m$. Existing sparse phase retrieval algorithms are usually first-order and hence converge at most linearly. Inspired by the hard thresholding pursuit (HTP) algorithm in compressed sensing, we propose an efficient second-order algorithm for sparse phase retrieval. Our proposed algorithm is theoretically guaranteed to give an exact sparse signal recovery in finite (in particular, at most $O(\log m + \log(\|\bm{x}^{\natural}\|_2/|x_{\min}^{\natural}|))$) steps, when $\{\bm{a}_i\}_{i=1}^{m}$ are i.i.d. standard Gaussian random vector with $m\sim O(s\log(n/s))$ and the initialization is in a neighborhood of the underlying sparse signal. Together with a spectral initialization, our algorithm is guaranteed to have an exact recovery from $O(s^2\log n)$ samples. Since the computational cost per iteration of our proposed algorithm is the same order as popular first-order algorithms, our algorithm is extremely efficient. Experimental results show that our algorithm can be several times faster than existing sparse phase retrieval algorithms.

math.NA↗

arXiv2021

Accelerated Structured Alternating Projections for Robust Spectrally Sparse Signal Recovery

Consider a spectrally sparse signal $\boldsymbol{x}$ that consists of $r$ complex sinusoids with or without damping. We study the robust recovery problem for the spectrally sparse signal under the fully observed setting, which is about recovering $\boldsymbol{x}$ and a sparse corruption vector $\boldsymbol{s}$ from their sum $\boldsymbol{z}=\boldsymbol{x}+\boldsymbol{s}$. In this paper, we exploit the low-rank property of the Hankel matrix formed by $\boldsymbol{x}$, and formulate the problem as the robust recovery of a corrupted low-rank Hankel matrix. We develop a highly efficient non-convex algorithm, coined Accelerated Structured Alternating Projections (ASAP). The high computational efficiency and low space complexity of ASAP are achieved by fast computations involving structured matrices, and a subspace projection method for accelerated low-rank approximation. Theoretical recovery guarantee with a linear convergence rate has been established for ASAP, under some mild assumptions on $\boldsymbol{x}$ and $\boldsymbol{s}$. Empirical performance comparisons on both synthetic and real-world data confirm the advantages of ASAP, in terms of computational efficiency and robustness aspects.

cs.IT↗

arXiv2021

Provable Near-Optimal Low-Multilinear-Rank Tensor Recovery

We consider the problem of recovering a low-multilinear-rank tensor from a small amount of linear measurements. We show that the Riemannian gradient algorithm initialized by one step of iterative hard thresholding can reconstruct an order-$d$ tensor of size $n\times\ldots\times n$ and multilinear rank $(r,\ldots,r)$ with high probability from only $O(nr^2 + r^{d+1})$ measurements, assuming $d$ is a constant. This sampling complexity is optimal in $n$, compared to existing results whose sampling complexities are all unnecessarily large in $n$. The analysis relies on the tensor restricted isometry property (TRIP) and the geometry of the manifold of all tensors with a fixed multilinear rank. High computational efficiency of our algorithm is also achieved by doing higher order singular value decomposition on intermediate small tensors of size only $2r\times \ldots\times 2r$ rather than on tensors of size $n\times \ldots\times n$ as usual.

math.NA↗

arXiv2020

Image Restoration: Structured Low Rank Matrix Framework for Piecewise Smooth Functions and Beyond

Recently, mapping a signal/image into a low rank Hankel/Toeplitz matrix has become an emerging alternative to the traditional sparse regularization, due to its ability to alleviate the basis mismatch between the true support in the continuous domain and the discrete grid. In this paper, we introduce a novel structured low rank matrix framework to restore piecewise smooth functions. Inspired by the total generalized variation to use sparse higher order derivatives, we derive that the Fourier samples of higher order derivatives satisfy an annihilation relation, resulting in a low rank multi-fold Hankel matrix. We further observe that the SVD of a low rank Hankel matrix corresponds to a tight wavelet frame system which can represent the image with sparse coefficients. Based on this observation, we also propose a wavelet frame analysis approach based continuous domain regularization model for the piecewise smooth image restoration. Finally, numerical results on image restoration tasks are presented as a proof-of-concept study to demonstrate that the proposed approach is compared favorably against several popular discrete regularization approaches and structured low rank matrix approaches.

math.NA↗

arXiv2020

An auto-parameter denoising method for nuclear magnetic resonance spectroscopy based on low-rank Hankel matrix

Nuclear Magnetic Resonance (NMR) spectroscopy, which is modeled as the sum of damped exponential signals, has become an indispensable tool in various scenarios, such as the structure and function determination, chemical analysis, and disease diagnosis. NMR spectroscopy signals, however, are usually corrupted by Gaussian noise in practice, raising difficulties in sequential analysis and quantification of the signals. The low-rank Hankel property plays an important role in the denoising issue, but selecting an appropriate parameter still remains a problem. In this work, we explore the effect of the regularization parameter of a convex optimization denoising method based on low-rank Hankel matrices for exponential signals corrupted by Gaussian noise. An accurate estimate on the spectral norm of weighted Hankel matrices is provided as a guidance to set the regularization parameter. The bound can be efficiently calculated since it only depends on the standard deviation of the noise and a constant. Aided by the bound, one can easily obtain an auto-setting regularization parameter to produce promising denoised results. Our experiments on synthetic and realistic NMR spectroscopy data demonstrate a superior denoising performance of our proposed approach in comparison with the typical Cadzow and the state-of-the-art QR decomposition methods, especially in the low signal-to-noise ratio regime.

physics.med-ph↗

arXiv2020

Data Driven Tight Frame for Compressed Sensing MRI Reconstruction via Off-the-Grid Regularization

Recently, the finite-rate-of-innovation (FRI) based continuous domain regularization is emerging as an alternative to the conventional on-the-grid sparse regularization for the compressed sensing (CS) due to its ability to alleviate the basis mismatch between the true support of the shape in the continuous domain and the discrete grid. In this paper, we propose a new off-the-grid regularization for the CS-MRI reconstruction. Following the recent works on two dimensional FRI, we assume that the discontinuities/edges of the image are localized in the zero level set of a band-limited periodic function. This assumption induces the linear dependencies among the Fourier samples of the gradient of the image, which leads to a low rank two-fold Hankel matrix. We further observe that the singular value decomposition of a low rank Hankel matrix corresponds to an adaptive tight frame system which can represent the image with sparse canonical coefficients. Based on this observation, we propose a data driven tight frame based off-the-grid regularization model for the CS-MRI reconstruction. To solve the nonconvex and nonsmooth model, a proximal alternating minimization algorithm with a guaranteed global convergence is adopted. Finally, the numerical experiments show that our proposed data driven tight frame based approach outperforms the existing approaches.

math.NA↗

arXiv2020

Fast Cadzow's Algorithm and a Gradient Variant

The Cadzow's algorithm is a signal denoising and recovery method which was designed for signals corresponding to low rank Hankel matrices. In this paper we first introduce a Fast Cadzow's algorithm which is developed by incorporating a novel subspace projection to reduce the high computational cost of the SVD in the Cadzow's algorithm. Then a Gradient method and a Fast Gradient method are proposed to address the non-decreasing MSE issue when applying the Cadzow's or Fast Cadzow's algorithm for signal denoising. Extensive empirical performance comparisons demonstrate that the proposed algorithms can complete the denoising and recovery tasks more efficiently and effectively.

math.NA↗

arXiv2019

Optimal low rank tensor recovery

We investigate the sample size requirement for exact recovery of a high order tensor of low rank from a subset of its entries. In the Tucker decomposition framework, we show that the Riemannian optimization algorithm with initial value obtained from a spectral method can reconstruct a tensor of size $n\times n \times\cdots \times n$ tensor of ranks $(r,\cdots,r)$ with high probability from as few as $O((r^d+dnr)\log(d))$ entries. In the case of order 3 tensor, the entries can be asymptotically as few as $O(nr)$ for a low rank large tensor. We show the theoretical guarantee condition for the recovery. The analysis relies on the tensor restricted isometry property (tensor RIP) and the curvature of the low rank tensor manifold. Our algorithm is computationally efficient and easy to implement. Numerical results verify that the algorithms are able to recover a low rank tensor from minimum number of measurements. The experiments on hyperspectral images recovery also show that our algorithm is capable of real world signal processing problems.

stat.ML↗

arXiv2019

Towards the optimal construction of a loss function without spurious local minima for solving quadratic equations

The problem of finding a vector $x$ which obeys a set of quadratic equations $|a_k^\top x|^2=y_k$, $k=1,\cdots,m$, plays an important role in many applications. In this paper we consider the case when both $x$ and $a_k$ are real-valued vectors of length $n$. A new loss function is constructed for this problem, which combines the smooth quadratic loss function with an activation function. Under the Gaussian measurement model, we establish that with high probability the target solution $x$ is the unique local minimizer (up to a global phase factor) of the new loss function provided $m\gtrsim n$. Moreover, the loss function always has a negative directional curvature around its saddle points.

cs.IT↗

arXiv2019

Scalable Incremental Nonconvex Optimization Approach for Phase Retrieval

We aim to find a solution $\bm{x}\in\mathbb{C}^n$ to a system of quadratic equations of the form $b_i=\lvert\bm{a}_i^*\bm{x}\rvert^2$, $i=1,2,\ldots,m$, e.g., the well-known NP-hard phase retrieval problem. As opposed to recently proposed state-of-the-art nonconvex methods, we revert to the semidefinite relaxation (SDR) PhaseLift convex formulation and propose a successive and incremental nonconvex optimization algorithm, termed as \texttt{IncrePR}, to indirectly minimize the resulting convex problem on the cone of positive semidefinite matrices. Our proposed method overcomes the excessive computational cost of typical SDP solvers as well as the need of a good initialization for typical nonconvex methods. For Gaussian measurements, which is usually needed for provable convergence of nonconvex methods, \texttt{IncrePR} with restart strategy outperforms state-of-the-art nonconvex solvers with a sharper phase transition of perfect recovery and typical convex solvers in terms of computational cost and storage. For more challenging structured (non-Gaussian) measurements often occurred in real applications, such as transmission matrix and oversampling Fourier transform, \texttt{IncrePR} with several restarts can be used to find a good initial guess. With further refinement by local nonconvex solvers, one can achieve a better solution than that obtained by applying nonconvex solvers directly when the number of measurements is relatively small. Extensive numerical tests are performed to demonstrate the effectiveness of the proposed method.

math.OC↗

arXiv2019

Fast Single Image Reflection Suppression via Convex Optimization

Removing undesired reflections from images taken through the glass is of great importance in computer vision. It serves as a means to enhance the image quality for aesthetic purposes as well as to preprocess images in machine learning and pattern recognition applications. We propose a convex model to suppress the reflection from a single input image. Our model implies a partial differential equation with gradient thresholding, which is solved efficiently using Discrete Cosine Transform. Extensive experiments on synthetic and real-world images demonstrate that our approach achieves desirable reflection suppression results and dramatically reduces the execution time.

cs.CV↗

arXiv2019

Large Scale 2D Spectral Compressed Sensing in Continuous Domain

We consider the problem of spectral compressed sensing in continuous domain, which aims to recover a 2-dimensional spectrally sparse signal from partially observed time samples. The signal is assumed to be a superposition of s complex sinusoids. We propose a semidefinite program for the 2D signal recovery problem. Our model is able to handle large scale 2D signals of size 500*500, whereas traditional approaches only handle signals of size around 20*20.

eess.SP↗

arXiv2019

Accelerated Alternating Projections for Robust Principal Component Analysis

We study robust PCA for the fully observed setting, which is about separating a low rank matrix $\boldsymbol{L}$ and a sparse matrix $\boldsymbol{S}$ from their sum $\boldsymbol{D}=\boldsymbol{L}+\boldsymbol{S}$. In this paper, a new algorithm, dubbed accelerated alternating projections, is introduced for robust PCA which significantly improves the computational efficiency of the existing alternating projections proposed in [Netrapalli, Praneeth, et al., 2014] when updating the low rank factor. The acceleration is achieved by first projecting a matrix onto some low dimensional subspace before obtaining a new estimate of the low rank matrix via truncated SVD. Exact recovery guarantee has been established which shows linear convergence of the proposed algorithm. Empirical performance evaluations establish the advantage of our algorithm over other state-of-the-art algorithms for robust PCA.

cs.IT↗

arXiv2019

Enhanced Expressive Power and Fast Training of Neural Networks by Random Projections

Random projections are able to perform dimension reduction efficiently for datasets with nonlinear low-dimensional structures. One well-known example is that random matrices embed sparse vectors into a low-dimensional subspace nearly isometrically, known as the restricted isometric property in compressed sensing. In this paper, we explore some applications of random projections in deep neural networks. We provide the expressive power of fully connected neural networks when the input data are sparse vectors or form a low-dimensional smooth manifold. We prove that the number of neurons required for approximating a Lipschitz function with a prescribed precision depends on the sparsity or the dimension of the manifold and weakly on the dimension of the input vector. The key in our proof is that random projections embed stably the set of sparse vectors or a low-dimensional smooth manifold into a low-dimensional subspace. Based on this fact, we also propose some new neural network models, where at each layer the input is first projected onto a low-dimensional subspace by a random projection and then the standard linear connection and non-linear activation are applied. In this way, the number of parameters in neural networks is significantly reduced, and therefore the training of neural networks can be accelerated without too much performance loss.

cs.LG↗

arXiv2018

Data-Driven Tight Frame for Cryo-EM Image Denoising and Conformational Classification

The cryo-electron microscope (cryo-EM) is increasingly popular these years. It helps to uncover the biological structures and functions of macromolecules. In this paper, we address image denoising problem in cryo-EM. Denoising the cryo-EM images can help to distinguish different molecular conformations and improve three dimensional reconstruction resolution. We introduce the use of data-driven tight frame (DDTF) algorithm for cryo-EM image denoising. The DDTF algorithm is closely related to the dictionary learning. The advantage of DDTF algorithm is that it is computationally efficient, and can well identify the texture and shape of images without using large data samples. Experimental results on cryo-EM image denoising and conformational classification demonstrate the power of DDTF algorithm for cryo-EM image denoising and classification.

stat.CO↗