Searcharxiv⌕ Search

arXiv subjects

Tong Zhang

Publications and source records attributed to Tong Zhang.

At least 685 records · Page 38Linked to original sources
arXiv2014

Communication Efficient Distributed Optimization using an Approximate Newton-type Method

We present a novel Newton-type method for distributed optimization, which is particularly well suited for stochastic optimization and learning problems. For quadratic objectives, the method enjoys a linear rate of convergence which provably \emph{improves} with the data size, requiring an essentially constant number of iterations under reasonable assumptions. We provide theoretical and empirical evidence of the advantages of our method compared to other approaches, such as one-shot parameter averaging and ADMM.

cs.LG↗
arXiv2014

Accelerating Minibatch Stochastic Gradient Descent using Stratified Sampling

Stochastic Gradient Descent (SGD) is a popular optimization method which has been applied to many important machine learning tasks such as Support Vector Machines and Deep Neural Networks. In order to parallelize SGD, minibatch training is often employed. The standard approach is to uniformly sample a minibatch at each step, which often leads to high variance. In this paper we propose a stratified sampling strategy, which divides the whole dataset into clusters with low within-cluster variance; we then take examples from these clusters using a stratified sampling technique. It is shown that the convergence rate can be significantly improved by the algorithm. Encouraging experimental results confirm the effectiveness of the proposed method.

stat.ML↗
arXiv2014

Quasiparticle scattering from topological crystalline insulator SnTe (001) surface states

Recently, the topological classification of electronic states has been extended to a new class of matter known as topological crystalline insulators. Similar to topological insulators, topological crystalline insulators also have spin-momentum locked surface states; but they only exist on specific crystal planes that are protected by crystal reflection symmetry. Here, we report an ultra-low temperature scanning tunneling microscopy and spectroscopy study on topological crystalline insulator SnTe nanoplates grown by molecular beam epitaxy. We observed quasiparticle interference patterns on the SnTe (001) surface that can be interpreted in terms of electron scattering from the four Fermi pockets of the topological crystalline insulator surface states in the first surface Brillouin zone. A quantitative analysis of the energy dispersion of the quasiparticle interference intensity shows two high energy features related to the crossing point beyond the Lifshitz transition when the two neighboring low energy surface bands near the point merge. A comparison between the experimental and computed quasiparticle interference patterns reveals possible spin texture of the surface states.

cond-mat.mes-hall↗
arXiv2014

Three-dimensional manipulation of electron beam phase space for seeding soft x-ray free-electron lasers

In this letter, a simple technique is proposed to induce strong density modulation into the electron beam with small energy modulation. By using the combination of a transversely dispersed electron beam and a wave-front tilted seed laser, three-dimensional manipulation of the electron beam phase space can be utilized to significantly enhance the micro-bunching of seeded free-electron laser schemes, which will improve the performance and extend the short-wavelength range of a single-stage seeded free-electron laser. Theoretical analysis and numerical simulations demonstrate the capability of the proposed technique in a soft x-ray free-electron laser.

physics.acc-ph↗
arXiv2014

Single-shot measurement of free-electron laser polarization at SDUV-FEL

In this paper, a division-of-amplitude photopolarimeter (DOAP) for measuring the polarization state of free-electron laser (FEL) pulse is described. The incident FEL beam is divided into four separate beams, and four Stokes parameters can be measured in a single-shot. In the crossed-planar undulators experiment at Shanghai deep ultraviolet FEL test facility, this DOAP instrument constructed in house responses accurately and timely while the polarization-state of fully coherent FEL pulses are switched, which is helpful for confirming the crossed-planar undulators technique for short-wavelength FELs.

physics.acc-ph↗
arXiv2014

Random design analysis of ridge regression

This work gives a simultaneous analysis of both the ordinary least squares estimator and the ridge regression estimator in the random design setting under mild assumptions on the covariate/response distributions. In particular, the analysis provides sharp results on the ``out-of-sample'' prediction error, as opposed to the ``in-sample'' (fixed design) error. The analysis also reveals the effect of errors in the estimated covariance structure, as well as the effect of modeling errors, neither of which effects are present in the fixed design setting. The proofs of the main results are based on a simple decomposition lemma combined with concentration inequalities for random vectors and matrices.

math.ST↗
arXiv2014

A Proximal Stochastic Gradient Method with Progressive Variance Reduction

We consider the problem of minimizing the sum of two convex functions: one is the average of a large number of smooth component functions, and the other is a general convex function that admits a simple proximal mapping. We assume the whole objective function is strongly convex. Such problems often arise in machine learning, known as regularized empirical risk minimization. We propose and analyze a new proximal stochastic gradient method, which uses a multi-stage scheme to progressively reduce the variance of the stochastic gradient. While each iteration of this algorithm has similar cost as the classical stochastic gradient method (or incremental gradient method), we show that the expected objective value converges to the optimum at a geometric rate. The overall complexity of this method is much lower than both the proximal full gradient method and the standard proximal stochastic gradient method.

math.OC↗
arXiv2014

Enhanced Precision Through Multiple Reads for LDPC Decoding in Flash Memories

Multiple reads of the same Flash memory cell with distinct word-line voltages provide enhanced precision for LDPC decoding. In this paper, the word-line voltages are optimized by maximizing the mutual information (MI) of the quantized channel. The enhanced precision from a few additional reads allows FER performance to approach that of full-precision soft information and enables an LDPC code to significantly outperform a BCH code. A constant-ratio constraint provides a significant simplification in the optimization with no noticeable loss in performance. For a well-designed LDPC code, the quantization that maximizes the mutual information also minimizes the frame error rate in our simulations. However, for an example LDPC code with a high error floor caused by small absorbing sets, the MMI quantization does not provide the lowest frame error rate. The best quantization in this case introduces more erasures than would be optimal for the channel MI in order to mitigate the absorbing sets of the poorly designed code. The paper also identifies a trade-off in LDPC code design when decoding is performed with multiple precision levels; the best code at one level of precision will typically not be the best code at a different level of precision.

cs.IT↗
arXiv2013

Sparse Recovery with Very Sparse Compressed Counting

Compressed sensing (sparse signal recovery) often encounters nonnegative data (e.g., images). Recently we developed the methodology of using (dense) Compressed Counting for recovering nonnegative K-sparse signals. In this paper, we adopt very sparse Compressed Counting for nonnegative signal recovery. Our design matrix is sampled from a maximally-skewed p-stable distribution (0 0, it suffices to use M= K/(1-exp(-gK) log N measurements, so that all coordinates can be recovered in one scan of the coordinates. If g = 1 (i.e., dense design), then M = K log N. If g= 1/K or 2/K (i.e., very sparse design), then M = 1.58K log N or M = 1.16K log N. This means the design matrix can be indeed very sparse at only a minor inflation of the sample complexity. Interestingly, as p->1, the required number of measurements is essentially M = 2.7K log N, provided g= 1/K. It turns out that this result is a general worst-case bound.

stat.ME↗
arXiv2013

Gradient Hard Thresholding Pursuit for Sparsity-Constrained Optimization

Hard Thresholding Pursuit (HTP) is an iterative greedy selection procedure for finding sparse solutions of underdetermined linear systems. This method has been shown to have strong theoretical guarantee and impressive numerical performance. In this paper, we generalize HTP from compressive sensing to a generic problem setup of sparsity-constrained convex optimization. The proposed algorithm iterates between a standard gradient descent step and a hard thresholding step with or without debiasing. We prove that our method enjoys the strong guarantees analogous to HTP in terms of rate of convergence and parameter estimation accuracy. Numerical evidences show that our method is superior to the state-of-the-art greedy selection methods in sparse logistic regression and sparse precision matrix estimation tasks.

cs.LG↗
arXiv2013

Geography of irregular Gorenstein 3-folds

In this paper, we study the explicit geography problem of irregular Gorenstein minimal 3-folds of general type. We generalize the classical Noether-Castelnuovo inequalities for irregular surfaces to irregular 3-folds according to the Albanese dimension.

math.AG↗
arXiv2013

Learning Pairwise Graphical Models with Nonlinear Sufficient Statistics

We investigate a generic problem of learning pairwise exponential family graphical models with pairwise sufficient statistics defined by a global mapping function, e.g., Mercer kernels. This subclass of pairwise graphical models allow us to flexibly capture complex interactions among variables beyond pairwise product. We propose two $\ell_1$-norm penalized maximum likelihood estimators to learn the model parameters from i.i.d. samples. The first one is a joint estimator which estimates all the parameters simultaneously. The second one is a node-wise conditional estimator which estimates the parameters individually for each node. For both estimators, we show that under proper conditions the extra flexibility gained in our model comes at almost no cost of statistical and computational efficiency. We demonstrate the advantages of our model over state-of-the-art methods on synthetic and real datasets.

stat.ML↗
arXiv2013

Fast Polarization Switching Demonstration Using Crossed-Planar Undulator in a Seeded Free Electron Laser

Fast polarization switching of light sources is required over a wide spectral range to investigate the symmetry of matter. In this Letter, we report the first experimental demonstration of the crossed-planar undulator technique at a seeded free-electron laser, which holds great promise for the full control and fast switching of the polarization of short-wavelength radiation. In the experiment, the polarization state of the coherent radiation at the 2nd harmonic of the seed laser is switched successfully. The experiment results confirm the theory, and pave the way for applying the crossed-planar undulator technique for the seeded X-ray free electron lasers.

physics.acc-ph↗
arXiv2013

Aggregation of Affine Estimators

We consider the problem of aggregating a general collection of affine estimators for fixed design regression. Relevant examples include some commonly used statistical estimators such as least squares, ridge and robust least squares estimators. Dalalyan and Salmon (2012) have established that, for this problem, exponentially weighted (EW) model selection aggregation leads to sharp oracle inequalities in expectation, but similar bounds in deviation were not previously known. While results indicate that the same aggregation scheme may not satisfy sharp oracle inequalities with high probability, we prove that a weaker notion of oracle inequality for EW that holds with high probability. Moreover, using a generalization of the newly introduced $Q$-aggregation scheme we also prove sharp oracle inequalities that hold with high probability. Finally, we apply our results to universal aggregation and show that our proposed estimator leads simultaneously to all the best known bounds for aggregation, including $\ell_q$-aggregation, $q \in (0,1)$, with high probability.

math.ST↗
arXiv2013

Accelerated Proximal Stochastic Dual Coordinate Ascent for Regularized Loss Minimization

We introduce a proximal version of the stochastic dual coordinate ascent method and show how to accelerate the method using an inner-outer iteration procedure. We analyze the runtime of the framework and obtain rates that improve state-of-the-art results for various key machine learning optimization problems including SVM, logistic regression, ridge regression, Lasso, and multiclass SVM. Experiments validate our theoretical findings.

stat.ML↗
arXiv2013

Compressed Counting Meets Compressed Sensing

Compressed sensing (sparse signal recovery) has been a popular and important research topic in recent years. By observing that natural signals are often nonnegative, we propose a new framework for nonnegative signal recovery using Compressed Counting (CC). CC is a technique built on maximally-skewed p-stable random projections originally developed for data stream computations. Our recovery procedure is computationally very efficient in that it requires only one linear scan of the coordinates. Our analysis demonstrates that, when 0 0 and C=pi/2 when p=0.5. In particular, when p->0 the required number of measurements is essentially M=K\log N, where K is the number of nonzero coordinates of the signal.

stat.ME↗
arXiv2013

High-dimensional Joint Sparsity Random Effects Model for Multi-task Learning

Joint sparsity regularization in multi-task learning has attracted much attention in recent years. The traditional convex formulation employs the group Lasso relaxation to achieve joint sparsity across tasks. Although this approach leads to a simple convex formulation, it suffers from several issues due to the looseness of the relaxation. To remedy this problem, we view jointly sparse multi-task learning as a specialized random effects model, and derive a convex relaxation approach that involves two steps. The first step learns the covariance matrix of the coefficients using a convex formulation which we refer to as sparse covariance coding; the second step solves a ridge regression problem with a sparse quadratic regularizer based on the covariance matrix obtained in the first step. It is shown that this approach produces an asymptotically optimal quadratic regularizer in the multitask learning setting when the number of tasks approaches infinity. Experimental results demonstrate that the convex formulation obtained via the proposed model significantly outperforms group Lasso (and related multi-stage formulations

cs.LG↗