arXiv subjects

Ping Li

Publications and source records attributed to Ping Li.

At least 325 records · Page 18Linked to original sources

arXiv2020

Distributed Primal-Dual Optimization for Online Multi-Task Learning

Conventional online multi-task learning algorithms suffer from two critical limitations: 1) Heavy communication caused by delivering high velocity of sequential data to a central machine; 2) Expensive runtime complexity for building task relatedness. To address these issues, in this paper we consider a setting where multiple tasks are geographically located in different places, where one task can synchronize data with others to leverage knowledge of related tasks. Specifically, we propose an adaptive primal-dual algorithm, which not only captures task-specific noise in adversarial learning but also carries out a projection-free update with runtime efficiency. Moreover, our model is well-suited to decentralized periodic-connected tasks as it allows the energy-starved or bandwidth-constraint tasks to postpone the update. Theoretical results demonstrate the convergence guarantee of our distributed algorithm with an optimal regret. Empirical results confirm that the proposed model is highly effective on various real-world datasets.

stat.ML↗

arXiv2020

The rainbow vertex-disconnection in graphs

Let $G$ be a nontrivial connected and vertex-colored graph. A subset $X$ of the vertex set of $G$ is called rainbow if any two vertices in $X$ have distinct colors. The graph $G$ is called \emph{rainbow vertex-disconnected} if for any two vertices $x$ and $y$ of $G$, there exists a vertex subset $S$ of $G$ such that when $x$ and $y$ are nonadjacent, $S$ is rainbow and $x$ and $y$ belong to different components of $G-S$; whereas when $x$ and $y$ are adjacent, $S+x$ or $S+y$ is rainbow and $x$ and $y$ belong to different components of $(G-xy)-S$. For a connected graph $G$, the \emph{rainbow vertex-disconnection number} of $G$, denoted by $rvd(G)$, is the minimum number of colors that are needed to make $G$ rainbow vertex-disconnected. In this paper, we characterize all graphs of order $n$ with rainbow vertex-disconnection number $k$ for $k\in\{1,2,n\}$, and determine the rainbow vertex-disconnection numbers of some special graphs. Moreover, we study the extremal problems on the number of edges of a connected graph $G$ with order $n$ and $rvd(G)=k$ for given integers $k$ and $n$ with $1\leq k\leq n$.

math.CO↗

arXiv2020

An Inverse-free Truncated Rayleigh-Ritz Method for Sparse Generalized Eigenvalue Problem

This paper considers the sparse generalized eigenvalue problem (SGEP), which aims to find the leading eigenvector with at most $k$ nonzero entries. SGEP naturally arises in many applications in machine learning, statistics, and scientific computing, for example, the sparse principal component analysis (SPCA), the sparse discriminant analysis (SDA), and the sparse canonical correlation analysis (SCCA). In this paper, we focus on the development of a three-stage algorithm named {\em inverse-free truncated Rayleigh-Ritz method} ({\em IFTRR}) to efficiently solve SGEP. In each iteration of IFTRR, only a small number of matrix-vector products is required. This makes IFTRR well-suited for large scale problems. Particularly, a new truncation strategy is proposed, which is able to find the support set of the leading eigenvector effectively. Theoretical results are developed to explain why IFTRR works well. Numerical simulations demonstrate the merits of IFTRR.

stat.ML↗

arXiv2020

Solving the Robust Matrix Completion Problem via a System of Nonlinear Equations

We consider the problem of robust matrix completion, which aims to recover a low rank matrix $L_*$ and a sparse matrix $S_*$ from incomplete observations of their sum $M=L_*+S_*\in\mathbb{R}^{m\times n}$. Algorithmically, the robust matrix completion problem is transformed into a problem of solving a system of nonlinear equations, and the alternative direction method is then used to solve the nonlinear equations. In addition, the algorithm is highly parallelizable and suitable for large scale problems. Theoretically, we characterize the sufficient conditions for when $L_*$ can be approximated by a low rank approximation of the observed $M_*$. And under proper assumptions, it is shown that the algorithm converges to the true solution linearly. Numerical simulations show that the simple method works as expected and is comparable with state-of-the-art methods.

stat.ML↗

arXiv2020

Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systems

Neural networks of ads systems usually take input from multiple resources, e.g., query-ad relevance, ad features and user portraits. These inputs are encoded into one-hot or multi-hot binary features, with typically only a tiny fraction of nonzero feature values per example. Deep learning models in online advertising industries can have terabyte-scale parameters that do not fit in the GPU memory nor the CPU main memory on a computing node. For example, a sponsored online advertising system can contain more than $10^{11}$ sparse features, making the neural network a massive model with around 10 TB parameters. In this paper, we introduce a distributed GPU hierarchical parameter server for massive scale deep learning ads systems. We propose a hierarchical workflow that utilizes GPU High-Bandwidth Memory, CPU main memory and SSD as 3-layer hierarchical storage. All the neural network training computations are contained in GPUs. Extensive experiments on real-world data confirm the effectiveness and the scalability of the proposed system. A 4-node hierarchical GPU parameter server can train a model more than 2X faster than a 150-node in-memory distributed parameter server in an MPI cluster. In addition, the price-performance ratio of our proposed system is 4-9 times better than an MPI-cluster solution.

cs.DC↗

arXiv2020

Meta-CoTGAN: A Meta Cooperative Training Paradigm for Improving Adversarial Text Generation

Training generative models that can generate high-quality text with sufficient diversity is an important open problem for Natural Language Generation (NLG) community. Recently, generative adversarial models have been applied extensively on text generation tasks, where the adversarially trained generators alleviate the exposure bias experienced by conventional maximum likelihood approaches and result in promising generation quality. However, due to the notorious defect of mode collapse for adversarial training, the adversarially trained generators face a quality-diversity trade-off, i.e., the generator models tend to sacrifice generation diversity severely for increasing generation quality. In this paper, we propose a novel approach which aims to improve the performance of adversarial text generation via efficiently decelerating mode collapse of the adversarial training. To this end, we introduce a cooperative training paradigm, where a language model is cooperatively trained with the generator and we utilize the language model to efficiently shape the data distribution of the generator against mode collapse. Moreover, instead of engaging the cooperative update for the generator in a principled way, we formulate a meta learning mechanism, where the cooperative update to the generator serves as a high level meta task, with an intuition of ensuring the parameters of the generator after the adversarial update would stay resistant against mode collapse. In the experiment, we demonstrate our proposed approach can efficiently slow down the pace of mode collapse for the adversarial text generators. Overall, our proposed method is able to outperform the baseline approaches with significant margins in terms of both generation quality and diversity in the testified domains.

cs.CL↗

arXiv2020

Nonnegative Hermitian vector bundles and Chern numbers

We show in this article that if a holomorphic vector bundle has a nonnegative Hermitian metric in the sense of Bott and Chern, which always exists on globally generated holomorphic vector bundles, then some special linear combinations of Chern forms are strongly nonnegative. This particularly implies that all the Chern numbers of such a holomorphic vector bundle are nonnegative and can be bounded below and above respectively by two special Chern numbers. As applications, we obtain a family of new results on compact connected complex manifolds which are homogeneous or can be holomorphically immersed into complex tori, some of which improve several classical results.

math.DG↗

arXiv2020

Two-dimensional transition metal oxides Mn2O3 realized quantum anomalous Hall effect

The quantum anomalous Hall effect is a intriguing topological nontrivial phase arising from spontaneous magnetization and spin-orbit coupling. However, the tremendously harsh realizing requirements of the quantum anomalous Hall effects in magnetic topological insulators of Cr or V-doped (Bi,Sb)2Te3 film, hinder its practical applications. Here, we use first principles calculations to predict that the three Mn2O3 structure is an intrinsic ferromagnetic Chern insulator. Remarkably, a quantum anomalous Hall phase of Chern number C = -2 is found, and there are two corresponding gapless chiral edge states appearing inside the bulk gap. More interestingly, only a small tensile strain is needed to induce the phase transition from Cmm2 and C222 phase to P6/mmm phase. Meanwhile, a topological quantum phase transition between a quantum anomalous Hall phase and a trivial insulating phase can be realize. The combination of these novel properties renders the two-dimensional ferromagnet a promising platform for high effciency electronic and spintronic applications.

physics.comp-ph↗

arXiv2020

Selective Convolutional Network: An Efficient Object Detector with Ignoring Background

It is well known that attention mechanisms can effectively improve the performance of many CNNs including object detectors. Instead of refining feature maps prevalently, we reduce the prohibitive computational complexity by a novel attempt at attention. Therefore, we introduce an efficient object detector called Selective Convolutional Network (SCN), which selectively calculates only on the locations that contain meaningful and conducive information. The basic idea is to exclude the insignificant background areas, which effectively reduces the computational cost especially during the feature extraction. To solve it, we design an elaborate structure with negligible overheads to guide the network where to look next. It's end-to-end trainable and easy-embedding. Without additional segmentation datasets, we explores two different train strategies including direct supervision and indirect supervision. Extensive experiments assess the performance on PASCAL VOC2007 and MS COCO detection datasets. Results show that SSD and Pelee integrated with our method averagely reduce the calculations in a range of 1/5 and 1/3 with slight loss of accuracy, demonstrating the feasibility of SCN.

cs.CV↗

arXiv2020

Structure-Feature based Graph Self-adaptive Pooling

Various methods to deal with graph data have been proposed in recent years. However, most of these methods focus on graph feature aggregation rather than graph pooling. Besides, the existing top-k selection graph pooling methods have a few problems. First, to construct the pooled graph topology, current top-k selection methods evaluate the importance of the node from a single perspective only, which is simplistic and unobjective. Second, the feature information of unselected nodes is directly lost during the pooling process, which inevitably leads to a massive loss of graph feature information. To solve these problems mentioned above, we propose a novel graph self-adaptive pooling method with the following objectives: (1) to construct a reasonable pooled graph topology, structure and feature information of the graph are considered simultaneously, which provide additional veracity and objectivity in node selection; and (2) to make the pooled nodes contain sufficiently effective graph information, node feature information is aggregated before discarding the unimportant nodes; thus, the selected nodes contain information from neighbor nodes, which can enhance the use of features of the unselected nodes. Experimental results on four different datasets demonstrate that our method is effective in graph classification and outperforms state-of-the-art graph pooling methods.

cs.SI↗

arXiv2020

Generalization Bounds for High-dimensional M-estimation under Sparsity Constraint

The $\ell_0$-constrained empirical risk minimization ($\ell_0$-ERM) is a promising tool for high-dimensional statistical estimation. The existing analysis of $\ell_0$-ERM estimator is mostly on parameter estimation and support recovery consistency. From the perspective of statistical learning, another fundamental question is how well the $\ell_0$-ERM estimator would perform on unseen samples. The answer to this question is important for understanding the learnability of such a non-convex (and also NP-hard) M-estimator but still relatively under explored. In this paper, we investigate this problem and develop a generalization theory for $\ell_0$-ERM. We establish, in both white-box and black-box statistical regimes, a set of generalization gap and excess risk bounds for $\ell_0$-ERM to characterize its sparse prediction and optimization capability. Our theory mainly reveals three findings: 1) tighter generalization bounds can be attained by $\ell_0$-ERM than those of $\ell_2$-ERM if the risk function is (with high probability) restricted strongly convex; 2) tighter uniform generalization bounds can be established for $\ell_0$-ERM than the conventional dense ERM; and 3) sparsity level invariant bounds can be established by imposing additional strong-signal conditions to ensure the stability of $\ell_0$-ERM. In light of these results, we further provide generalization guarantees for the Iterative Hard Thresholding (IHT) algorithm which serves as one of the most popular greedy pursuit methods for approximately solving $\ell_0$-ERM. Numerical evidence is provided to confirm our theoretical predictions when implied to sparsity-constrained linear regression and logistic regression models.

math.ST↗

arXiv2020

Rainbow monochromatic $k$-edge-connection colorings of graphs

A path in an edge-colored graph is called a monochromatic path if all edges of the path have a same color. We call $k$ paths $P_1,\cdots,P_k$ rainbow monochromatic paths if every $P_i$ is monochromatic and for any two $i\neq j$, $P_i$ and $P_j$ have different colors. An edge-coloring of a graph $G$ is said to be a rainbow monochromatic $k$-edge-connection coloring (or $RMC_k$-coloring for short) if every two distinct vertices of $G$ are connected by at least $k$ rainbow monochromatic paths. We use $rmc_k(G)$ to denote the maximum number of colors that ensures $G$ has an $RMC_k$-coloring, and this number is called the rainbow monochromatic $k$-edge-connection number. We prove the existence of $RMC_k$-colorings of graphs, and then give some bounds of $rmc_k(G)$ and present some graphs whose $rmc_k(G)$ reaches the lower bound. We also obtain the threshold function for $rmc_k(G(n,p))\geq f(n)$, where $\lfloor\frac{n}{2}\rfloor> k\geq 1$.

math.CO↗

arXiv2019

On some applications of Gauduchon metrics

We apply the existence and special properties of Gauduchon metrics to give several applications. The first one is concerned with the implications of algebro-geometric nature under the existence of a Hermitian metric with nonnegative holomorphic sectional curvature. The second one is to show the non-existence of holomorphic sections on Hermitian vector bundles under certain conditions. The third one is to give a restriction on the $\partial\bar{\partial}$-closedness of some real $(n-1,n-1)$-forms on compact complex manifolds.

math.DG↗

arXiv2019

Outlier Detection and Data Clustering via Innovation Search

The idea of Innovation Search was proposed as a data clustering method in which the directions of innovation were utilized to compute the adjacency matrix and it was shown that Innovation Pursuit can notably outperform the self representation based subspace clustering methods. In this paper, we present a new discovery that the directions of innovation can be used to design a provable and strong robust (to outlier) PCA method. The proposed approach, dubbed iSearch, uses the direction search optimization problem to compute an optimal direction corresponding to each data point. iSearch utilizes the directions of innovation to measure the innovation of the data points and it identifies the outliers as the most innovative data points. Analytical performance guarantees are derived for the proposed robust PCA method under different models for the distribution of the outliers including randomly distributed outliers, clustered outliers, and linearly dependent outliers. In addition, we study the problem of outlier detection in a union of subspaces and it is shown that iSearch provably recovers the span of the inliers when the inliers lie in a union of subspaces. Moreover, we present theoretical studies which show that the proposed measure of innovation remains stable in the presence of noise and the performance of iSearch is robust to noisy data. In the challenging scenarios in which the outliers are close to each other or they are close to the span of the inliers, iSearch is shown to remarkably outperform most of the existing methods. The presented method shows that the directions of innovation are useful representation of the data which can be used to perform both data clustering and outlier detection.

stat.ML↗

arXiv2019

Fully spin-polarized quadratic non-Dirac bands realized quantum anomalous Hall effect

The quantum anomalous Hall effect is a intriguing quantum state which exhibits the chiral edge states in the absence of magnetic field. While the search for quantum anomalous Hall insulators is still active, the researchers mainly search for the systems containing magnetic atom. Here, based on first-principles density functional theory, we predict a new family of chern insulators with fully spin-polarized quadratic px;y non-Dirac bands in the alkali earth metal BaX (X = Si, Ge, Sn) system. We show that BaX monolayer has a half-metallic ferromagnetic ground state. The ferromagnetism is mainly originated from the p orbitals of Si, Ge and Sn atoms. The 2D BaSn monolayer exhibits a large magnetocrystalline anisotropic energy of 12.20 meV/cell and a nontrivial band gap of 159.10 meV. Interestingly, both the spin polarization of the chiral edge currents and the sign of Chern number can be tuned by doping. Furthermore, the 4 % compressive strain can drive structural phase transition but the nontrivial topological properties remain reserve in the 2D BaX systems. Our findings not only extend the novel concepts but also provide fascinating opportunities for the realization of quantum anomalous Hall effect experimentally.

cond-mat.mes-hall↗

arXiv2019

Monochromatic disconnection of graphs

For an edge-colored graph $G$, we call an edge-cut $M$ of $G$ monochromatic if the edges of $M$ are colored with a same color. The graph $G$ is called monochromatically disconnected if any two distinct vertices of $G$ are separated by a monochromatic edge-cut. For a connected graph $G$, the monochromatic disconnection number, denoted by $md(G)$, of $G$ is the maximum number of colors that are needed in order to make $G$ monochromatically disconnected. We will show that almost all graphs have monochromatic disconnection numbers equal to 1. We also obtain the Nordhaus-Gaddum-type results for $md(G)$.

math.CO↗

arXiv2019

Graph Analysis and Graph Pooling in the Spatial Domain

The spatial convolution layer which is widely used in the Graph Neural Networks (GNNs) aggregates the feature vector of each node with the feature vectors of its neighboring nodes. The GNN is not aware of the locations of the nodes in the global structure of the graph and when the local structures corresponding to different nodes are similar to each other, the convolution layer maps all those nodes to similar or same feature vectors in the continuous feature space. Therefore, the GNN cannot distinguish two graphs if their difference is not in their local structures. In addition, when the nodes are not labeled/attributed the convolution layers can fail to distinguish even different local structures. In this paper, we propose an effective solution to address this problem of the GNNs. The proposed approach leverages a spatial representation of the graph which makes the neural network aware of the differences between the nodes and also their locations in the graph. The spatial representation which is equivalent to a point-cloud representation of the graph is obtained by a graph embedding method. Using the proposed approach, the local feature extractor of the GNN distinguishes similar local structures in different locations of the graph and the GNN infers the topological structure of the graph from the spatial distribution of the locally extracted feature vectors. Moreover, the spatial representation is utilized to simplify the graph down-sampling problem. A new graph pooling method is proposed and it is shown that the proposed pooling method achieves competitive or better results in comparison with the state-of-the-art methods.

cs.LG↗

arXiv2019

Quantum anomalous Hall effect in two dimensional Janus Mn2Cl3Br3 with large magnetic anisotropy energy

The quantum anomalous Hall (QAH) effect have been experimentally observed in magnetically-doped topological insulators. However, the QAH effect only at extremely low temperatures due to the weak magnetic coupling, small band gap and low carrier mobility. Here, based on first-principles density functional theory, we predict that the Janus Mn2Cl3Br3 is high Curie temperature ferromagnet that host the QAH phase. Furthermore, we find that it is a Dirac half-metal characterized by a Dirac cone in one spin channel with carrier mobilities comparable to freestanding germanene and an large band gap in other spin channel. Simultaneously, when the spin-orbital coupling interaction is considered, the Janus Mn2Cl3Br3 exhibit lager magnetic anisotropic energy of 11.89 meV/cell and a nontrivial band gap. More interestingly, both the Chern number sign and the chiral edge current are tuned by changing the direction of magnetization. Our finding would suggest the possibility of not only realized the QAH effect but also designed the flow direction of the edge current.

cond-mat.mtrl-sci↗