arXiv subjects

Michael K. Ng

Publications and source records attributed to Michael K. Ng.

90 records · Page 5Linked to original sources

arXiv2021

A parallel-in-time two-sided preconditioning for all-at-once system from a non-local evolutionary equation with weakly singular kernel

In this paper, we study a parallel-in-time (PinT) algorithm for all-at-once system from a non-local evolutionary equation with weakly singular kernel where the temporal term involves a non-local convolution with a weakly singular kernel and the spatial term is the usual Laplacian operator with variable coefficients. We propose to use a two-sided preconditioning technique for the all-at-once discretization of the equation. Our preconditioner is constructed by replacing the variable diffusion coefficients with a constant coefficient to obtain a constant-coefficient all-at-once matrix. We split a square root of the constant Laplacian operator out of the constant-coefficient all-at-once matrix as a right preconditioner and take the remaining part as a left preconditioner, which constitutes our two-sided preconditioning. Exploiting the diagonalizability of the constant-Laplacian matrix and the triangular Toeplitz structure of the temporal discretization matrix, we obtain efficient representations of inverses of the right and the left preconditioners, because of which the iterative solution can be fast updated in a PinT manner. Theoretically, the condition number of the two-sided preconditioned matrix is proven to be uniformly bounded by a constant independent of the matrix size. To the best of our knowledge, for the non-local evolutionary equation with variable coefficients, this is the first attempt to develop a PinT preconditioning technique that has fast and exact implementation and that the corresponding preconditioned system has a uniformly bounded condition number. Numerical results are reported to confirm the efficiency of the proposed two-sided preconditioning technique.

math.NA↗

arXiv2020

Low Rank Pure Quaternion Approximation for Pure Quaternion Matrices

Quaternion matrices are employed successfully in many color image processing applications. In particular, a pure quaternion matrix can be used to represent red, green and blue channels of color images. A low-rank approximation for a pure quaternion matrix can be obtained by using the quaternion singular value decomposition. However, this approximation is not optimal in the sense that the resulting low-rank approximation matrix may not be pure quaternion, i.e., the low-rank matrix contains real component which is not useful for the representation of a color image. The main contribution of this paper is to find an optimal rank-$r$ pure quaternion matrix approximation for a pure quaternion matrix (a color image). Our idea is to use a projection on a low-rank quaternion matrix manifold and a projection on a quaternion matrix with zero real component, and develop an alternating projections algorithm to find such optimal low-rank pure quaternion matrix approximation. The convergence of the projection algorithm can be established by showing that the low-rank quaternion matrix manifold and the zero real component quaternion matrix manifold has a non-trivial intersection point. Numerical examples on synthetic pure quaternion matrices and color images are presented to illustrate the projection algorithm can find optimal low-rank pure quaternion approximation for pure quaternion matrices or color images.

math.NA↗

arXiv2020

Riemannian Conjugate Gradient Descent Method for Third-Order Tensor Completion

The goal of tensor completion is to fill in missing entries of a partially known tensor under a low-rank constraint. In this paper, we mainly study low rank third-order tensor completion problems by using Riemannian optimization methods on the smooth manifold. Here the tensor rank is defined to be a set of matrix ranks where the matrices are the slices of the transformed tensor obtained by applying the Fourier-related transformation onto the tubes of the original tensor. We show that with suitable incoherence conditions on the underlying low rank tensor, the proposed Riemannian optimization method is guaranteed to converge and find such low rank tensor with a high probability. In addition, numbers of sample entries required for solving low rank tensor completion problem under different initialized methods are studied and derived. Numerical examples for both synthetic and image data sets are reported to demonstrate the proposed method is able to recover low rank tensors.

math.OC↗

arXiv2020

Tangent Space Based Alternating Projections for Nonnegative Low Rank Matrix Approximation

In this paper, we develop a new alternating projection method to compute nonnegative low rank matrix approximation for nonnegative matrices. In the nonnegative low rank matrix approximation method, the projection onto the manifold of fixed rank matrices can be expensive as the singular value decomposition is required. We propose to use the tangent space of the point in the manifold to approximate the projection onto the manifold in order to reduce the computational cost. We show that the sequence generated by the alternating projections onto the tangent spaces of the fixed rank matrices manifold and the nonnegative matrix manifold, converge linearly to a point in the intersection of the two manifolds where the convergent point is sufficiently close to optimal solutions. This convergence result based inexact projection onto the manifold is new and is not studied in the literature. Numerical examples in data clustering, pattern recognition and hyperspectral data analysis are given to demonstrate that the performance of the proposed method is better than that of nonnegative matrix factorization methods in terms of computational time and accuracy.

cs.LG↗

arXiv2020

Tensorizing GAN with High-Order Pooling for Alzheimer's Disease Assessment

It is of great significance to apply deep learning for the early diagnosis of Alzheimer's Disease (AD). In this work, a novel tensorizing GAN with high-order pooling is proposed to assess Mild Cognitive Impairment (MCI) and AD. By tensorizing a three-player cooperative game based framework, the proposed model can benefit from the structural information of the brain. By incorporating the high-order pooling scheme into the classifier, the proposed model can make full use of the second-order statistics of the holistic Magnetic Resonance Imaging (MRI) images. To the best of our knowledge, the proposed Tensor-train, High-pooling and Semi-supervised learning based GAN (THS-GAN) is the first work to deal with classification on MRI images for AD diagnosis. Extensive experimental results on Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset are reported to demonstrate that the proposed THS-GAN achieves superior performance compared with existing methods, and to show that both tensor-train and high-order pooling can enhance classification performance. The visualization of generated samples also shows that the proposed model can generate plausible samples for semi-supervised learning purpose.

cs.LG↗

arXiv2020

Tensor train rank minimization with nonlocal self-similarity for tensor completion

The tensor train (TT) rank has received increasing attention in tensor completion due to its ability to capture the global correlation of high-order tensors ($\textrm{order} >3$). For third order visual data, direct TT rank minimization has not exploited the potential of TT rank for high-order tensors. The TT rank minimization accompany with \emph{ket augmentation}, which transforms a lower-order tensor (e.g., visual data) into a higher-order tensor, suffers from serious block-artifacts. To tackle this issue, we suggest the TT rank minimization with nonlocal self-similarity for tensor completion by simultaneously exploring the spatial, temporal/spectral, and nonlocal redundancy in visual data. More precisely, the TT rank minimization is performed on a formed higher-order tensor called group by stacking similar cubes, which naturally and fully takes advantage of the ability of TT rank for high-order tensors. Moreover, the perturbation analysis for the TT low-rankness of each group is established. We develop the alternating direction method of multipliers tailored for the specific structure to solve the proposed model. Extensive experiments demonstrate that the proposed method is superior to several existing state-of-the-art methods in terms of both qualitative and quantitative measures.

cs.CV↗

arXiv2020

New Formulation and Computation for Generalized Singular Values of Grassman Matrix Pair

In this paper, we derive new model formulations for computing generalized singular values of a Grassman matrix pair. These new formulations make use of truncated filter matrices to locate the $i$-th generalized singular value of a Grassman matrix pair. The resulting matrix optimization problems can be solved by using numerical methods involving Newton's method on Grassmann manifold. Numerical examples on synthetic data sets and gene expression data sets are reported to demonstrate the high accuracy and the fast computation of the proposed new ormulations for computing arbitrary generalized singular value of Grassman matrix pair.

math.NA↗

arXiv2020

Fast Alternating Projections on Manifolds Based on Tangent Spaces

In this paper, we study alternating projections on nontangential manifolds based on the tangent spaces. The main motivation is that the projection of a point onto a manifold can be computational expensive. We propose to use the tangent space of the point in the manifold to approximate the projection onto the manifold in order to reduce the computational cost. We show that the sequence generated by alternating projections on two nontangential manifolds based on tangent spaces, converges linearly to a point in the intersection of the two manifolds where the convergent point is close to the optimal solution. Numerical examples for nonnegative low rank matrix approximation and low rank image quaternion matrix (color image) approximation, are given to demonstrate that the performance of the proposed method is better than that of the classical alternating projection method in terms of computational time.

math.NA↗

arXiv2020

Fast and High-order Accuracy Numerical Methods for Time-Dependent Nonlocal Problems in $\mathbb{R}^2

In this paper, we study the Crank-Nicolson method for temporal dimension and the piecewise quadratic polynomial collocation method for spatial dimensions of time-dependent nonlocal problems. The new theoretical results of such discretization are that the proposed numerical method is unconditionally stable and its global truncation error is of $\mathcal{O}\left(τ^2+h^{4-γ}\right)$ with $0<γ<1$, where $τ$ and $h$ are the discretization sizes in the temporal and spatial dimensions respectively. Also we develop the conjugate gradient squared method to solving the resulting discretized nonsymmetric and indefinite systems arising from time-dependent nonlocal problems including two-dimensional cases. By using additive and multiplicative Cauchy kernels in non-local problems, structured coefficient matrix-vector multiplication can be performed efficiently in the conjugate gradient squared iteration. Numerical examples are given to illustrate our theoretical results and demonstrate that the computational cost of the proposed method is of $O(M \log M)$ operations where $M$ is the number of collocation points.

math.NA↗

arXiv2020

Singular Vectors From Singular Values

In the recent paper \cite{1}, Denton et al. provided the eigenvector-eigenvalue identity for Hermitian matrices, and a survey was also given for such identity in the literature. The main aim of this paper is to present the identity related to singular vectors and singular values of a general matrix.

math.NA↗

arXiv2020

Bilinear Constraint based ADMM for Mixed Poisson-Gaussian Noise Removal

In this paper, we propose new operator-splitting algorithms for the total variation regularized infimal convolution (TV-IC) model [4] in order to remove mixed Poisson-Gaussian(MPG) noise. In the existing splitting algorithm for TV-IC, an inner loop by Newton method had to be adopted for one nonlinear optimization subproblem, which increased the computation cost per outer loop. By introducing a new bilinear constraint and applying the alternating direction method of multipliers (ADMM), all subproblems of the proposed algorithms named as BCA (short for Bilinear Constraint based ADMM algorithm) and BCAf(short for a variant of BCA with fully splitting form) can be very efficiently solved; especially for the proposed BCAf, they can be calculated without any inner iterations. Under mild conditions, the convergence of the proposed BCA is investigated. Numerically, compared to existing primal-dual algorithms for the TV-IC model, the proposed algorithms, with fewer tunable parameters, converge much faster and produce comparable results meanwhile.

math.OC↗

arXiv2020

Framelet Representation of Tensor Nuclear Norm for Third-Order Tensor Completion

The main aim of this paper is to develop a framelet representation of the tensor nuclear norm for third-order tensor completion. In the literature, the tensor nuclear norm can be computed by using tensor singular value decomposition based on the discrete Fourier transform matrix, and tensor completion can be performed by the minimization of the tensor nuclear norm which is the relaxation of the sum of matrix ranks from all Fourier transformed matrix frontal slices. These Fourier transformed matrix frontal slices are obtained by applying the discrete Fourier transform on the tubes of the original tensor. In this paper, we propose to employ the framelet representation of each tube so that a framelet transformed tensor can be constructed. Because of framelet basis redundancy, the representation of each tube is sparsely represented. When the matrix slices of the original tensor are highly correlated, we expect the corresponding sum of matrix ranks from all framelet transformed matrix frontal slices would be small, and the resulting tensor completion can be performed much better. The proposed minimization model is convex and global minimizers can be obtained. Numerical results on several types of multi-dimensional data (videos, multispectral images, and magnetic resonance imaging data) have tested and shown that the proposed method outperformed the other testing methods.

eess.IV↗

arXiv2019

Orthogonal Nonnegative Tucker Decomposition

In this paper, we study the nonnegative tensor data and propose an orthogonal nonnegative Tucker decomposition (ONTD). We discuss some properties of ONTD and develop a convex relaxation algorithm of the augmented Lagrangian function to solve the optimization problem. The convergence of the algorithm is given. We employ ONTD on the image data sets from the real world applications including face recognition, image representation, hyperspectral unmixing. Numerical results are shown to illustrate the effectiveness of the proposed algorithm.

stat.ML↗

arXiv2019

Robust Tensor Completion Using Transformed Tensor SVD

In this paper, we study robust tensor completion by using transformed tensor singular value decomposition (SVD), which employs unitary transform matrices instead of discrete Fourier transform matrix that is used in the traditional tensor SVD. The main motivation is that a lower tubal rank tensor can be obtained by using other unitary transform matrices than that by using discrete Fourier transform matrix. This would be more effective for robust tensor completion. Experimental results for hyperspectral, video and face datasets have shown that the recovery performance for the robust tensor completion problem by using transformed tensor SVD is better in PSNR than that by using Fourier transform and other robust tensor completion methods.

cs.LG↗

arXiv2019

An efficient second-order convergent scheme for one-side space fractional diffusion equations with variable coefficients

In this paper, a second order finite difference scheme is investigated for time-dependent one-side space fractional diffusion equations with variable coefficients. The existing schemes for the equation with variable coefficients have temporal convergence rate no better than second order and spatial convergence rate no better than first order, theoretically. In the presented scheme, the Crank-Nicolson temporal discretization and a second-order weighted-and-shifted Grünwald-Letnikov spatial discretization are employed. Theoretically, the unconditional stability and the second-order convergence in time and space of the proposed scheme are established under some conditions on the diffusion coefficients. Moreover, a Toeplitz preconditioner is proposed for linear systems arising from the proposed scheme. The condition number of the preconditioned matrix is proven to be bounded by a constant independent of the discretization step-sizes so that the Krylov subspace solver for the preconditioned linear systems converges linearly. Numerical results are reported to show the convergence rate and the efficiency of the proposed scheme.

math.NA↗

arXiv2017

Parallel Active Subspace Decomposition for Scalable and Efficient Tensor Robust Principal Component Analysis

Tensor robust principal component analysis (TRPCA) has received a substantial amount of attention in various fields. Most existing methods, normally relying on tensor nuclear norm minimization, need to pay an expensive computational cost due to multiple singular value decompositions (SVDs) at each iteration. To overcome the drawback, we propose a scalable and efficient method, named Parallel Active Subspace Decomposition (PASD), which divides the unfolding along each mode of the tensor into a columnwise orthonormal matrix (active subspace) and another small-size matrix in parallel. Such a transformation leads to a nonconvex optimization problem in which the scale of nulcear norm minimization is generally much smaller than that in the original problem. Furthermore, we introduce an alternating direction method of multipliers (ADMM) method to solve the reformulated problem and provide rigorous analyses for its convergence and suboptimality. Experimental results on synthetic and real-world data show that our algorithm is more accurate than the state-of-the-art approaches, and is orders of magnitude faster.

math.NA↗

arXiv2017

Exact Tensor Completion from Sparsely Corrupted Observations via Convex Optimization

This paper conducts a rigorous analysis for provable estimation of multidimensional arrays, in particular third-order tensors, from a random subset of its corrupted entries. Our study rests heavily on a recently proposed tensor algebraic framework in which we can obtain tensor singular value decomposition (t-SVD) that is similar to the SVD for matrices, and define a new notion of tensor rank referred to as the tubal rank. We prove that by simply solving a convex program, which minimizes a weighted combination of tubal nuclear norm, a convex surrogate for the tubal rank, and the $\ell_1$-norm, one can recover an incoherent tensor exactly with overwhelming probability, provided that its tubal rank is not too large and that the corruptions are reasonably sparse. Interestingly, our result includes the recovery guarantees for the problems of tensor completion (TC) and tensor principal component analysis (TRPCA) under the same algebraic setup as special cases. An alternating direction method of multipliers (ADMM) algorithm is presented to solve this optimization problem. Numerical experiments verify our theory and real-world applications demonstrate the effectiveness of our algorithm.

cs.LG↗

arXiv2017

Sparse Kernel Canonical Correlation Analysis via $\ell_1$-regularization

Canonical correlation analysis (CCA) is a multivariate statistical technique for finding the linear relationship between two sets of variables. The kernel generalization of CCA named kernel CCA has been proposed to find nonlinear relations between datasets. Despite their wide usage, they have one common limitation that is the lack of sparsity in their solution. In this paper, we consider sparse kernel CCA and propose a novel sparse kernel CCA algorithm (SKCCA). Our algorithm is based on a relationship between kernel CCA and least squares. Sparsity of the dual transformations is introduced by penalizing the $\ell_{1}$-norm of dual vectors. Experiments demonstrate that our algorithm not only performs well in computing sparse dual transformations but also can alleviate the over-fitting problem of kernel CCA.

stat.ML↗