arXiv subjects

Thomas Hofmann

Publications and source records attributed to Thomas Hofmann.

151 records · Page 9Linked to original sources

arXiv2015

A Variance Reduced Stochastic Newton Method

Quasi-Newton methods are widely used in practise for convex loss minimization problems. These methods exhibit good empirical performance on a wide variety of tasks and enjoy super-linear convergence to the optimal solution. For large-scale learning problems, stochastic Quasi-Newton methods have been recently proposed. However, these typically only achieve sub-linear convergence rates and have not been shown to consistently perform well in practice since noisy Hessian approximations can exacerbate the effect of high-variance stochastic gradient estimates. In this work we propose Vite, a novel stochastic Quasi-Newton algorithm that uses an existing first-order technique to reduce this variance. Without exploiting the specific form of the approximate Hessian, we show that Vite reaches the optimum at a geometric rate with a constant step-size when dealing with smooth strongly convex functions. Empirically, we demonstrate improvements over existing stochastic Quasi-Newton and variance reduced stochastic gradient methods.

cs.LG↗

arXiv2014

Communication-Efficient Distributed Dual Coordinate Ascent

Communication remains the most significant bottleneck in the performance of distributed optimization algorithms for large-scale machine learning. In this paper, we propose a communication-efficient framework, CoCoA, that uses local computation in a primal-dual setting to dramatically reduce the amount of necessary communication. We provide a strong convergence rate analysis for this class of algorithms, as well as experiments on real-world distributed datasets with implementations in Spark. In our experiments, we find that as compared to state-of-the-art mini-batch versions of SGD and SDCA algorithms, CoCoA converges to the same .001-accurate solution quality on average 25x as quickly.

cs.LG↗

arXiv2014

Chemical and Crystallographic Characterization of the Tip Apex in Scanning Probe Microscopy

The apex atom of a W scanning probe tip reveals a non-spherical charge distribution as probed by a CO molecule bonded to a Cu(111) surface [Welker et al. Science, 336, 444 (2012)]. Three high-symmetry images were observed and related to three low-index crystallographic directions of the W bcc crystal. Open questions remained, however, including the verification that the tip was indeed W-terminated, and whether this method can be easily applied to distinguish other atomic species. In this work, we investigate bulk Cu and Fe tips. In both cases we can associate our data with the fcc (Cu) and bcc (Fe) crystal structures. A model is presented, based on the partial filling of d orbitals, to relate the AFM images to the angular orientation of the tip structure.

cond-mat.mes-hall↗

arXiv2013

Probabilistic Latent Semantic Analysis

Probabilistic Latent Semantic Analysis is a novel statistical technique for the analysis of two-mode and co-occurrence data, which has applications in information retrieval and filtering, natural language processing, machine learning from text, and in related areas. Compared to standard Latent Semantic Analysis which stems from linear algebra and performs a Singular Value Decomposition of co-occurrence tables, the proposed method is based on a mixture decomposition derived from a latent class model. This results in a more principled approach which has a solid foundation in statistics. In order to avoid overfitting, we propose a widely applicable generalization of maximum likelihood model fitting by tempered EM. Our approach yields substantial and consistent improvements over Latent Semantic Analysis in a number of experiments.

cs.LG↗

arXiv2012

Exponential Families for Conditional Random Fields

In this paper we de ne conditional random elds in reproducing kernel Hilbert spaces and show connections to Gaussian Process classi cation. More speci cally, we prove decomposition results for undirected graphical models and we give constructions for kernels. Finally we present e cient means of solving the optimization problem using reduced rank decompositions and we show how stationarity can be exploited e ciently in the optimization process.

cs.LG↗

arXiv2010

Preparation of light-atom tips for Scanning Probe Microscopy by explosive delamination

To obtain maximal resolution in STM and AFM, the size of the protruding tip orbital has to be minimized. Beryllium as tip material is a promising candidate for enhanced resolution because a beryllium atom has just four electrons, leading to a small covalent radius of only 96 pm. Besides that, beryllium is conductive and has a high elastic modulus, which is a necessity for a stable tip apex. However beryllium tips that are prepared ex situ, are covered with a robust oxide layer, which cannot be removed by just heating the tip. Here we present a successful preparation method that combines the heating of the tip by field emission and a mild collision with a clean metal plate. That method yields a clean, oxide-free tip surface as proven by a work function of as deduced from a current-distance curve. Additionally, a STM image of the Si-(111)-(7x7) is presented to prove the single-atom termination of the beryllium tip.

cond-mat.mes-hall↗

arXiv2008

Kernel methods in machine learning

We review machine learning methods employing positive definite kernels. These methods formulate learning and estimation problems in a reproducing kernel Hilbert space (RKHS) of functions defined on the data domain, expanded in terms of a kernel. Working in linear spaces of function has the benefit of facilitating the construction and analysis of learning algorithms while at the same time allowing large classes of functions. The latter include nonlinear functions as well as functions defined on nonvectorial data. We cover a wide range of methods, ranging from binary classifiers to sophisticated methods for estimation with structured data.

math.ST↗