Searcharxiv⌕ Search

arXiv subjects

Kai Zheng

Publications and source records attributed to Kai Zheng.

At least 181 records · Page 10Linked to original sources
arXiv2019

Equipping Experts/Bandits with Long-term Memory

We propose the first reduction-based approach to obtaining long-term memory guarantees for online learning in the sense of Bousquet and Warmuth, 2002, by reducing the problem to achieving typical switching regret. Specifically, for the classical expert problem with $K$ actions and $T$ rounds, using our framework we develop various algorithms with a regret bound of order $\mathcal{O}(\sqrt{T(S\ln T + n \ln K)})$ compared to any sequence of experts with $S-1$ switches among $n \leq \min\{S, K\}$ distinct experts. In addition, by plugging specific adaptive algorithms into our framework we also achieve the best of both stochastic and adversarial environments simultaneously. This resolves an open problem of Warmuth and Koolen, 2014. Furthermore, we extend our results to the sparse multi-armed bandit setting and show both negative and positive results for long-term memory guarantees. As a side result, our lower bound also implies that sparse losses do not help improve the worst-case regret for contextual bandits, a sharp contrast with the non-contextual case.

cs.LG↗
arXiv2019

Learning Transferable Self-attentive Representations for Action Recognition in Untrimmed Videos with Weak Supervision

Action recognition in videos has attracted a lot of attention in the past decade. In order to learn robust models, previous methods usually assume videos are trimmed as short sequences and require ground-truth annotations of each video frame/sequence, which is quite costly and time-consuming. In this paper, given only video-level annotations, we propose a novel weakly supervised framework to simultaneously locate action frames as well as recognize actions in untrimmed videos. Our proposed framework consists of two major components. First, for action frame localization, we take advantage of the self-attention mechanism to weight each frame, such that the influence of background frames can be effectively eliminated. Second, considering that there are trimmed videos publicly available and also they contain useful information to leverage, we present an additional module to transfer the knowledge from trimmed videos for improving the classification performance in untrimmed ones. Extensive experiments are conducted on two benchmark datasets (i.e., THUMOS14 and ActivityNet1.3), and experimental results clearly corroborate the efficacy of our method.

cs.CV↗
arXiv2018

Efficient Online Portfolio with Logarithmic Regret

We study the decades-old problem of online portfolio management and propose the first algorithm with logarithmic regret that is not based on Cover's Universal Portfolio algorithm and admits much faster implementation. Specifically Universal Portfolio enjoys optimal regret $\mathcal{O}(N\ln T)$ for $N$ financial instruments over $T$ rounds, but requires log-concave sampling and has a large polynomial running time. Our algorithm, on the other hand, ensures a slightly larger but still logarithmic regret of $\mathcal{O}(N^2(\ln T)^4)$, and is based on the well-studied Online Mirror Descent framework with a novel regularizer that can be implemented via standard optimization methods in time $\mathcal{O}(TN^{2.5})$ per round. The regret of all other existing works is either polynomial in $T$ or has a potentially unbounded factor such as the inverse of the smallest price relative.

cs.LG↗
arXiv2018

ProbeSim: Scalable Single-Source and Top-k SimRank Computations on Dynamic Graphs

Single-source and top-$k$ SimRank queries are two important types of similarity search in graphs with numerous applications in web mining, social network analysis, spam detection, etc. A plethora of techniques have been proposed for these two types of queries, but very few can efficiently support similarity search over large dynamic graphs, due to either significant preprocessing time or large space overheads. This paper presents ProbeSim, an index-free algorithm for single-source and top-$k$ SimRank queries that provides a non-trivial theoretical guarantee in the absolute error of query results. ProbeSim estimates SimRank similarities without precomputing any indexing structures, and thus can naturally support real-time SimRank queries on dynamic graphs. Besides the theoretical guarantee, ProbeSim also offers satisfying practical efficiency and effectiveness due to several non-trivial optimizations. We conduct extensive experiments on a number of benchmark datasets, which demonstrate that our solutions significantly outperform the existing methods in terms of efficiency and effectiveness. Notably, our experiments include the first empirical study that evaluates the effectiveness of SimRank algorithms on graphs with billion edges, using the idea of pooling.

cs.DB↗
arXiv2018

PABO: Mitigating Congestion via Packet Bounce in Data Center Networks

In today's data center, a diverse mix of throughput-sensitive long flows and delay-sensitive short flows are commonly presented in shallow-buffered switches. Long flows could potentially block the transmission of delay-sensitive short flows, leading to degraded performance. Congestion can also be caused by the synchronization of multiple TCP connections for short flows, as typically seen in the partition/aggregate traffic pattern. While multiple end-to-end transport-layer solutions have been proposed, none of them have tackled the real challenge: reliable transmission in the network. In this paper, we fill this gap by presenting PABO -- a novel link-layer design that can mitigate congestion by temporarily bouncing packets to upstream switches. PABO's design fulfills the following goals: i) providing per-flow based flow control on the link layer, ii) handling transient congestion without the intervention of end devices, and iii) gradually back propagating the congestion signal to the source when the network is not capable to handle the congestion.Experiment results show that PABO can provide prominent advantage of mitigating transient congestions and can achieve significant gain on end-to-end delay.

cs.NI↗
arXiv2018

Conic singularities metrics with prescribed scalar curvature: a priori estimates for normal crossing divisors

The purpose of this paper is to prove the a priori estimates for constant scalar curvature Kaehler metrics with conic singularities along normal crossing divisors. The zero order estimates are proved by a reformulated version of Alexandrov's maximum principle. The higher order estimates follow from Chen-Cheng's frame work, equipped with new techniques to handle the singularities. Finally, we extend these estimates to the twisted equations.

math.DG↗
arXiv2017

RUM: network Representation learning throUgh Multi-level structural information preservation

We have witnessed the discovery of many techniques for network representation learning in recent years, ranging from encoding the context in random walks to embedding the lower order connections, to finding latent space representations with auto-encoders. However, existing techniques are looking mostly into the local structures in a network, while higher-level properties such as global community structures are often neglected. We propose a novel network representations learning model framework called RUM (network Representation learning throUgh Multi-level structural information preservation). In RUM, we incorporate three essential aspects of a node that capture a network's characteristics in multiple levels: a node's affiliated local triads, its neighborhood relationships, and its global community affiliations. Therefore the framework explicitly and comprehensively preserves the structural information of a network, extending the encoding process both to the local end of the structural information spectrum and to the global end. The framework is also flexible enough to take various community discovery algorithms as its preprocessor. Empirical results show that the representations learned by RUM have demonstrated substantial performance advantages in real-life tasks.

cs.LG↗
arXiv2017

Geodesics in the space of Kahler cone metrics, II. Uniqueness of constant scalar curvature Kahler cone metrics

This is a continuation of the previous articles on Kahler cone metrics. In this article, we introduce weighted function spaces and provide a self-contained treatment on cone angles in the whole interval $(0,1]$. We first construct geodesics in the space of Kahler cone metrics (cone geodesics). We next determine the very detailed asymptotic behaviour of constant scalar curvature Kahler (cscK) cone metrics, which leads to the reductivity of the automorphism group. Then we establish the linear theory for the Lichnerowicz operator, which immediately implies the openness of the path deforming the cone angles of cscK cone metrics. Finally, we address the problem on the uniqueness of cscK cone metrics and show that the cscK cone metric is unique up to automorphisms.

math.DG↗
arXiv2017

End-to-end Learning for Short Text Expansion

Effectively making sense of short texts is a critical task for many real world applications such as search engines, social media services, and recommender systems. The task is particularly challenging as a short text contains very sparse information, often too sparse for a machine learning algorithm to pick up useful signals. A common practice for analyzing short text is to first expand it with external information, which is usually harvested from a large collection of longer texts. In literature, short text expansion has been done with all kinds of heuristics. We propose an end-to-end solution that automatically learns how to expand short text to optimize a given learning task. A novel deep memory network is proposed to automatically find relevant information from a collection of longer documents and reformulate the short text through a gating mechanism. Using short text classification as a demonstrating task, we show that the deep memory network significantly outperforms classical text expansion methods with comprehensive experiments on real world data sets.

cs.CL↗
arXiv2017

Generalization Bounds of SGLD for Non-convex Learning: Two Theoretical Viewpoints

Algorithm-dependent generalization error bounds are central to statistical learning theory. A learning algorithm may use a large hypothesis space, but the limited number of iterations controls its model capacity and generalization error. The impacts of stochastic gradient methods on generalization error for non-convex learning problems not only have important theoretical consequences, but are also critical to generalization errors of deep learning. In this paper, we study the generalization errors of Stochastic Gradient Langevin Dynamics (SGLD) with non-convex objectives. Two theories are proposed with non-asymptotic discrete-time analysis, using Stability and PAC-Bayesian results respectively. The stability-based theory obtains a bound of $O\left(\frac{1}{n}L\sqrt{βT_k}\right)$, where $L$ is uniform Lipschitz parameter, $β$ is inverse temperature, and $T_k$ is aggregated step sizes. For PAC-Bayesian theory, though the bound has a slower $O(1/\sqrt{n})$ rate, the contribution of each step is shown with an exponentially decaying factor by imposing $\ell^2$ regularization, and the uniform Lipschitz constant is also replaced by actual norms of gradients along trajectory. Our bounds have no implicit dependence on dimensions, norms or other capacity measures of parameter, which elegantly characterizes the phenomenon of "Fast Training Guarantees Generalization" in non-convex settings. This is the first algorithm-dependent result with reasonable dependence on aggregated step sizes for non-convex learning, and has important implications to statistical learning aspects of stochastic gradient methods in complicated models such as deep learning.

cs.LG↗
arXiv2017

Collect at Once, Use Effectively: Making Non-interactive Locally Private Learning Possible

Non-interactive Local Differential Privacy (LDP) requires data analysts to collect data from users through noisy channel at once. In this paper, we extend the frontiers of Non-interactive LDP learning and estimation from several aspects. For learning with smooth generalized linear losses, we propose an approximate stochastic gradient oracle estimated from non-interactive LDP channel, using Chebyshev expansion. Combined with inexact gradient methods, we obtain an efficient algorithm with quasi-polynomial sample complexity bound. For the high-dimensional world, we discover that under $\ell_2$-norm assumption on data points, high-dimensional sparse linear regression and mean estimation can be achieved with logarithmic dependence on dimension, using random projection and approximate recovery. We also extend our methods to Kernel Ridge Regression. Our work is the first one that makes learning and estimation possible for a broad range of learning tasks under non-interactive LDP model.

cs.LG↗
arXiv2017

Efficient Private ERM for Smooth Objectives

In this paper, we consider efficient differentially private empirical risk minimization from the viewpoint of optimization algorithms. For strongly convex and smooth objectives, we prove that gradient descent with output perturbation not only achieves nearly optimal utility, but also significantly improves the running time of previous state-of-the-art private optimization algorithms, for both $ε$-DP and $(ε, δ)$-DP. For non-convex but smooth objectives, we propose an RRPSGD (Random Round Private Stochastic Gradient Descent) algorithm, which provably converges to a stationary point with privacy guarantee. Besides the expected utility bounds, we also provide guarantees in high probability form. Experiments demonstrate that our algorithm consistently outperforms existing method in both utility and running time.

cs.LG↗
arXiv2017

Generalized Matsushima's theorem and Kähler-Einstein cone metrics

In this paper, we prove Matsushima's theorem for Kähler-Einstein metrics on a Fano manifold with cone singularities along a smooth divisor that is not necessarily proportional to the anti-canonical class. We then give an alternative proof of uniqueness of Kähler-Einstein cone metrics by the continuity method. Moreover, our method provides an existence theorem of Kähler-Einstein cone metrics with respect to conic Ding functional.

math.DG↗
arXiv2017

Construction of constant scalar curvature Kähler cone metrics

Over a compact Kähler manifold, we provide a Fredholm alternative result for the Lichnerowicz operator associated to a Kähler metric with conic singularities along a divisor. We deduce several existence results of constant scalar curvature Kähler metrics with conic singularities: existence result under small deformations of Kähler classes, existence result over a Fano manifold, existence result over certain ruled manifolds. In this last case, we consider the projectivisation of a parabolic stable holomorphic bundle. This leads us to prove that the existing Hermitian-Einstein metric on this bundle enjoys a regularity property along the divisor on the base.

math.DG↗
arXiv2016

Uniqueness of constant scalar curvature Kähler metrics with cone singularities, I: Reductivity

The aim of this paper is to investigate uniqueness of conic constant scalar curvature Kaehler (cscK) metrics, when the cone angle is less than $π$. We introduce a new Hölder space called $\cC^{4,\a,\b}$ to study the regularities of this fourth order elliptic equation, and prove that any $\cC^{2,\a,\b}$ conic cscK metric is indeed of class $\cC^{4,\a,\b}$. Finally, the reductivity is established by a careful study of the conic Lichnerowicz operator.

math.DG↗
arXiv2016

On the geodesic problem for the Dirichlet metric and the Ebin metric on the space of Sasakian metrics

We study the geodesic equation for the Dirichlet (gradient) metric in the space of Kaehler potentials. We first solve the initial value problem for the geodesic equation of the combination metric, including the gradient metric. We then discuss a comparison theorem between it and the Calabi metric. As geometric motivation of the combination metric, we find that the Ebin metric restricted to the space of type II deformations of a Sasakian structure is the sum of the Calabi metric and the gradient metric.

math.DG↗
arXiv2015

LazyCtrl: Scalable Network Control for Cloud Data Centers

The advent of software defined networking enables flexible, reliable and feature-rich control planes for data center networks. However, the tight coupling of centralized control and complete visibility leads to a wide range of issues among which scalability has risen to prominence. To address this, we present LazyCtrl, a novel hybrid control plane design for data center networks where network control is carried out by distributed control mechanisms inside independent groups of switches while complemented with a global controller. Our design is motivated by the observation that data center traffic is usually highly skewed and thus edge switches can be grouped according to traffic locality. LazyCtrl aims at bringing laziness to the global controller by dynamically devolving most of the control tasks to independent switch groups to process frequent intra-group events near datapaths while handling rare inter-group or other specified events by the controller. We implement LazyCtrl and build a prototype based on Open vSwich and Floodlight. Trace-driven experiments on our prototype show that an effective switch grouping is easy to maintain in multi-tenant clouds and the central controller can be significantly shielded by staying lazy, with its workload reduced by up to 82%.

cs.NI↗