Searcharxiv⌕ Search

arXiv subjects

Zuoqiang Shi

Publications and source records attributed to Zuoqiang Shi.

At least 91 records · Page 5Linked to original sources
arXiv2018

Deep Neural Nets with Interpolating Function as Output Activation

We replace the output layer of deep neural nets, typically the softmax function, by a novel interpolating function. And we propose end-to-end training and testing algorithms for this new architecture. Compared to classical neural nets with softmax function as output activation, the surrogate with interpolating function as output activation combines advantages of both deep and manifold learning. The new framework demonstrates the following major advantages: First, it is better applicable to the case with insufficient training data. Second, it significantly improves the generalization accuracy on a wide variety of networks. The algorithm is implemented in PyTorch, and code will be made publicly available.

cs.LG↗
arXiv2018

Scalable low dimensional manifold model in the reconstruction of noisy and incomplete hyperspectral images

We present a scalable low dimensional manifold model for the reconstruction of noisy and incomplete hyperspectral images. The model is based on the observation that the spatial-spectral blocks of a hyperspectral image typically lie close to a collection of low dimensional manifolds. To emphasize this, the dimension of the manifold is directly used as a regularizer in a variational functional, which is solved efficiently by alternating direction of minimization and weighted nonlocal Laplacian. Unlike general 3D images, the same similarity matrix can be shared across all spectral bands for a hyperspectral image, therefore the resulting algorithm is much more scalable than that for general 3D data. Numerical experiments on the reconstruction of hyperspectral images from sparse and noisy sampling demonstrate the superiority of our proposed algorithm in terms of both speed and accuracy.

cs.CV↗
arXiv2018

Weighted Nonlocal Total Variation in Image Processing

In this paper, a novel weighted nonlocal total variation (WNTV) method is proposed. Compared to the classical nonlocal total variation methods, our method modifies the energy functional to introduce a weight to balance between the labeled sets and unlabeled sets. With extensive numerical examples in semi-supervised clustering, image inpainting and image colorization, we demonstrate that WNTV provides an effective and efficient method in many image processing and machine learning problems.

cs.CV↗
arXiv2018

Deep Residual Learning and PDEs on Manifold

In this paper, we formulate the deep residual network (ResNet) as a control problem of transport equation. In ResNet, the transport equation is solved along the characteristics. Based on this observation, deep neural network is closely related to the control problem of PDEs on manifold. We propose several models based on transport equation and Hamilton-Jacobi equation. The discretization of these PDEs on point cloud is also discussed.

cs.IT↗
arXiv2017

A Flow Model of Neural Networks

Based on a natural connection between ResNet and transport equation or its characteristic equation, we propose a continuous flow model for both ResNet and plain net. Through this continuous model, a ResNet can be explicitly constructed as a refinement of a plain net. The flow model provides an alternative perspective to understand phenomena in deep neural networks, such as why it is necessary and sufficient to use 2-layer blocks in ResNets, why deeper is better, and why ResNets are even deeper, and so on. It also opens a gate to bring in more tools from the huge area of differential equations.

cs.LG↗
arXiv2016

Convergence of the Point Integral method for Poisson equation on point cloud

The Laplace-Beltrami operator (LBO) is a fundamental object associated to Riemannian manifolds, which encodes all intrinsic geometry of the manifolds and has many desirable properties. Recently, we proposed a novel numerical method, Point Integral method (PIM), to discretize the Laplace-Beltrami operator on point clouds \cite{LSS}. In this paper, we analyze the convergence of Point Integral method (PIM) for Poisson equation with Neumann boundary condition on submanifolds isometrically embedded in Euclidean spaces.

math.NA↗
arXiv2016

A convergent point integral method for isotropic elliptic equations on point cloud

In this paper, we propose a numerical method to solve isotropic elliptic equations on point cloud by generalizing the point integral method. The idea of the point integral method is to approximate the differential operators by integral operators and discretize the corresponding integral equation on point cloud. The key step is to get the integral approximation. In this paper, with proper kernel function, we get an integral approximation for the elliptic operators with isotropic coefficients. Moreover, the integral approximation has been proved to keep the coercivity of the original elliptic operator. The convergence of the point integral method is also proved.

math.NA↗
arXiv2016

A Harmonic Extension Approach for Collaborative Ranking

We present a new perspective on graph-based methods for collaborative ranking for recommender systems. Unlike user-based or item-based methods that compute a weighted average of ratings given by the nearest neighbors, or low-rank approximation methods using convex optimization and the nuclear norm, we formulate matrix completion as a series of semi-supervised learning problems, and propagate the known ratings to the missing ones on the user-user or item-item graph globally. The semi-supervised learning problems are expressed as Laplace-Beltrami equations on a manifold, or namely, harmonic extension, and can be discretized by a point integral method. We show that our approach does not impose a low-rank Euclidean subspace on the data points, but instead minimizes the dimension of the underlying manifold. Our method, named LDM (low dimensional manifold), turns out to be particularly effective in generating rankings of items, showing decent computational efficiency and robust ranking quality compared to state-of-the-art methods.

cs.LG↗
arXiv2015

Harmonic Extension

In this paper, we consider the harmonic extension problem, which is widely used in many applications of machine learning. We find that the transitional method of graph Laplacian fails to produce a good approximation of the classical harmonic function. To tackle this problem, we propose a new method called the point integral method (PIM). We consider the harmonic extension problem from the point of view of solving PDEs on manifolds. The basic idea of the PIM method is to approximate the harmonicity using an integral equation, which is easy to be discretized from points. Based on the integral equation, we explain the reason why the transitional graph Laplacian may fail to approximate the harmonicity in the classical sense and propose a different approach which we call the volume constraint method (VCM). Theoretically, both the PIM and the VCM computes a harmonic function with convergence guarantees, and practically, they are both simple, which amount to solve a linear system. One important application of the harmonic extension in machine learning is semi-supervised learning. We run a popular semi-supervised learning algorithm by Zhu et al. over a couple of well-known datasets and compare the performance of the aforementioned approaches. Our experiments show the PIM performs the best.

cs.LG↗
arXiv2015

Point Integral Method for Solving Poisson-type Equations on Manifolds from Point Clouds with Convergence Guarantees

Partial differential equations (PDE) on manifolds arise in many areas, including mathematics and many applied fields. Among all kinds of PDEs, the Poisson-type equations including the standard Poisson equation and the related eigenproblem of the Laplace-Beltrami operator are of the most important. Due to the complicated geometrical structure of the manifold, it is difficult to get efficient numerical method to solve PDE on manifold. In the paper, we propose a method called point integral method (PIM) to solve the Poisson-type equations from point clouds with convergence guarantees. In PIM, the key idea is to derive the integral equations which approximates the Poisson-type equations and contains no derivatives but only the values of the unknown function. The latter makes the integral equation easy to be approximated from point cloud. In the paper, we explain the derivation of the integral equations, describe the point integral method and its implementation, and present the numerical experiments to demonstrate the convergence of PIM.

math.NA↗
arXiv2015

Sparse Time-Frequency decomposition for multiple signals with same frequencies

In this paper, we consider multiple signals sharing same instantaneous frequencies. This kind of data is very common in scientific and engineering problems. To take advantage of this special structure, we modify our data-driven time-frequency analysis by updating the instantaneous frequencies simultaneously. Moreover, based on the simultaneously sparsity approximation and fast Fourier transform, some efficient algorithms is developed. Since the information of multiple signals is used, this method is very robust to the perturbation of noise. And it is applicable to the general nonperiodic signals even with missing samples or outliers. Several synthetic and real signals are used to test this method. The performances of this method are very promising.

cs.IT↗
arXiv2015

Convergence of Laplacian spectra from random samples

Eigenvectors and eigenvalues of discrete graph Laplacians are often used for manifold learning and nonlinear dimensionality reduction. It was previously proved by Belkin and Niyogi that the eigenvectors and eigenvalues of the graph Laplacian converge to the eigenfunctions and eigenvalues of the Laplace-Beltrami operator of the manifold in the limit of infinitely many data points sampled independently from the uniform distribution over the manifold. Recently, we introduced Point Integral method (PIM) to solve elliptic equations and corresponding eigenvalue problem on point clouds. We have established a unified framework to approximate the elliptic differential operators on point clouds. In this paper, we prove that the eigenvectors and eigenvalues obtained by PIM converge in the limit of infinitely many random samples independently from a distribution (not necessarily to be uniform distribution). Moreover, one estimate of the rate of the convergence is also given.

cs.IT↗
arXiv2015

Convergence of Laplacian Spectra from Point Clouds

The spectral structure of the Laplacian-Beltrami operator (LBO) on manifolds has been widely used in many applications, include spectral clustering, dimensionality reduction, mesh smoothing, compression and editing, shape segmentation, matching and parameterization, and so on. Typically, the underlying Riemannian manifold is unknown and often given by a set of sample points. The spectral structure of the LBO is estimated from some discrete Laplace operator constructed from this set of sample points. In our previous papers, we proposed the point integral method to discretize the LBO from point clouds, which is also capable to solve the eigenproblem. Then one fundmental issue is the convergence of the eigensystem of the discrete Laplacian to that of the LBO. In this paper, for compact manifolds isometrically embedded in Euclidean spaces possibly with boundary, we show that the eigenvalues and the eigenvectors obtained by the point integral method converges to the eigenvalues and the eigenfunctions of the LBO with the Neumann boundary, and in addition, we give an estimate of the convergence rate. This result provides a solid mathematical foundation for the point integral method in the computation of Laplacian spectra from point clouds.

math.NA↗
arXiv2015

Enforce the Dirichlet boundary condition by volume constraint in Point Integral method

Recently, Shi and Sun proposed Point Integral method (PIM) to discretize Laplace-Beltrami operator on point cloud. In PIM, Neumann boundary is nature, but Dirichlet boundary needs some special treatment. In our previous work, we use Robin boundary to approximate Dirichlet boundary. In this paper, we introduce another approach to deal with the Dirichlet boundary condition in point integral method using the volume constraint proposed by Du et.al.

math.NA↗
arXiv2015

On the Uniqueness of Sparse Time-Frequency Representation of Multiscale Data

In this paper, we analyze the uniqueness of the sparse time frequency decomposition and investigate the efficiency of the nonlinear matching pursuit method. Under the assumption of scale separation, we show that the sparse time frequency decomposition is unique up to an error that is determined by the scale separation property of the signal. We further show that the unique decomposition can be obtained approximately by the sparse time frequency decomposition using nonlinear matching pursuit.

cs.IT↗
arXiv2015

Extracting a shape function for a signal with intra-wave frequency modulation

In this paper, we consider signals with intra-wave frequency modulation. To handle this kind of signals effectively, we generalize our data-driven time-frequency analysis by using a shape function to describe the intra-wave frequency modulation. The idea of using a shape function in time-frequency analysis was first proposed by Wu. A shape function could be any periodic function. Based on this model, we propose to solve an optimization problem to extract the shape function. By exploring the fact that s is a periodic function, we can identify certain low rank structure of the signal. This structure enables us to extract the shape function from the signal. To test the robustness of our method, we apply our method on several synthetic and real signals. The results are very encouraging.

cs.IT↗
arXiv2014

Sparse Time-Frequency decomposition by dictionary learning

In this paper, we propose a time-frequency analysis method to obtain instantaneous frequencies and the corresponding decomposition by solving an optimization problem. In this optimization problem, the basis to decompose the signal is not known. Instead, it is adapted to the signal and is determined as part of the optimization problem. In this sense, this optimization problem can be seen as a dictionary learning problem. This dictionary learning problem is solved by using the Augmented Lagrangian Multiplier method (ALM) iteratively. We further accelerate the convergence of the ALM method in each iteration by using the fast wavelet transform. We apply our method to decompose several signals, including signals with poor scale separation, signals with outliers and polluted by noise and a real signal. The results show that this method can give accurate recovery of both the instantaneous frequencies and the intrinsic mode functions.

cs.IT↗