arXiv subjects

John Anderson

Publications and source records attributed to John Anderson.

34 records · Page 2Linked to original sources

arXiv2020

Stability and instability of traveling wave solutions to nonlinear wave equations

In this paper, we study the stability and instability of plane wave solutions to semilinear systems of wave equations satisfying the null condition. We identify a condition which allows us to prove the global nonlinear asymptotic stability of the plane wave. The proof of global stability requires us to analyze the geometry of the interaction between the background plane wave and the perturbation. When this condition is not met, we are able to prove linear instability assuming an additional genericity condition. The linear instability is shown using a geometric optics ansatz.

math.AP↗

arXiv2020

Zero-Shot Heterogeneous Transfer Learning from Recommender Systems to Cold-Start Search Retrieval

Many recent advances in neural information retrieval models, which predict top-K items given a query, learn directly from a large training set of (query, item) pairs. However, they are often insufficient when there are many previously unseen (query, item) combinations, often referred to as the cold start problem. Furthermore, the search system can be biased towards items that are frequently shown to a query previously, also known as the 'rich get richer' (a.k.a. feedback loop) problem. In light of these problems, we observed that most online content platforms have both a search and a recommender system that, while having heterogeneous input spaces, can be connected through their common output item space and a shared semantic representation. In this paper, we propose a new Zero-Shot Heterogeneous Transfer Learning framework that transfers learned knowledge from the recommender system component to improve the search component of a content platform. First, it learns representations of items and their natural-language features by predicting (item, item) correlation graphs derived from the recommender system as an auxiliary task. Then, the learned representations are transferred to solve the target search retrieval task, performing query-to-item prediction without having seen any (query, item) pairs in training. We conduct online and offline experiments on one of the world's largest search and recommender systems from Google, and present the results and lessons learned. We demonstrate that the proposed approach can achieve high performance on offline search retrieval tasks, and more importantly, achieved significant improvements on relevance and user interactions over the highly-optimized production system in online experiments.

cs.LG↗

arXiv2020

Neural Collaborative Filtering vs. Matrix Factorization Revisited

Embedding based models have been the state of the art in collaborative filtering for over a decade. Traditionally, the dot product or higher order equivalents have been used to combine two or more embeddings, e.g., most notably in matrix factorization. In recent years, it was suggested to replace the dot product with a learned similarity e.g. using a multilayer perceptron (MLP). This approach is often referred to as neural collaborative filtering (NCF). In this work, we revisit the experiments of the NCF paper that popularized learned similarities using MLPs. First, we show that with a proper hyperparameter selection, a simple dot product substantially outperforms the proposed learned similarities. Second, while a MLP can in theory approximate any function, we show that it is non-trivial to learn a dot product with an MLP. Finally, we discuss practical issues that arise when applying MLP based similarities and show that MLPs are too costly to use for item recommendation in production environments while dot products allow to apply very efficient retrieval algorithms. We conclude that MLPs should be used with care as embedding combiner and that dot products might be a better default choice.

cs.IR↗

arXiv2020

Superbloom: Bloom filter meets Transformer

We extend the idea of word pieces in natural language models to machine learning tasks on opaque ids. This is achieved by applying hash functions to map each id to multiple hash tokens in a much smaller space, similarly to a Bloom filter. We show that by applying a multi-layer Transformer to these Bloom filter digests, we are able to obtain models with high accuracy. They outperform models of a similar size without hashing and, to a large degree, models of a much larger size trained using sampled softmax with the same computational budget. Our key observation is that it is important to use a multi-layer Transformer for Bloom filter digests to remove ambiguity in the hashed input. We believe this provides an alternative method to solving problems with large vocabulary size.

cs.LG↗

arXiv2020

Sensitivity Analysis in the Dupire Local Volatility Model with Tensorflow

In a recent paper, we have demonstrated how the affinity between TPUs and multi-dimensional financial simulation resulted in fast Monte Carlo simulations that could be setup in a few lines of python Tensorflow code. We also presented a major benefit from writing high performance simulations in an automated differentiation language such as Tensorflow: a single line of code enabled us to estimate sensitivities, i.e. the rate of change in price of financial instrument with respect to another input such as the interest rate, the current price of the underlying, or volatility. Such sensitivities (otherwise known as the famous financial "Greeks") are fundamental for risk assessment and risk mitigation. In the present follow-up short paper, we extend the developments exposed in our previous work about the use of Tensor Processing Units and Tensorflow for TPUs.

cs.DC↗

arXiv2020

Tensor Processing Units for Financial Monte Carlo

Monte Carlo methods are critical to many routines in quantitative finance such as derivatives pricing, hedging and risk metrics. Unfortunately, Monte Carlo methods are very computationally expensive when it comes to running simulations in high-dimensional state spaces where they are still a method of choice in the financial industry. Recently, Tensor Processing Units (TPUs) have provided considerable speedups and decreased the cost of running Stochastic Gradient Descent (SGD) in Deep Learning. After highlighting computational similarities between training neural networks with SGD and simulating stochastic processes, we ask in the present paper whether TPUs are accurate, fast and simple enough to use for financial Monte Carlo. Through a theoretical reminder of the key properties of such methods and thorough empirical experiments we examine the fitness of TPUs for option pricing, hedging and risk metrics computation. In particular we demonstrate that, in spite of the use of mixed precision, TPUs still provide accurate estimators which are fast to compute when compared to GPUs. We also show that the Tensorflow programming model for TPUs is elegant, expressive and simplifies automated differentiation.

cs.DC↗

arXiv2019

High-resolution imaging on TPUs

The rapid evolution of artificial intelligence (AI) is leading to a new generation of hardware accelerators optimized for deep learning. Some of the designs of these accelerators are general enough to allow their use for other computationally intensive tasks beyond AI. Cloud tensor processing units (TPUs) are one such example. Here, we demonstrate a novel approach using TensorFlow on Cloud TPUs to implement a high-resolution imaging technique called full-waveform inversion. Higher-order numerical stencils leverage the efficient matrix multiplication offered by the Cloud TPU, and the halo exchange benefits from the dedicated high-speed interchip connection. The performance is competitive when compared with Tesla V100 graphics processing units and shows promise for future computation- and memory-intensive imaging applications.

cs.CE↗

arXiv2019

High Performance Monte Carlo Simulation of Ising Model on TPU Clusters

Large-scale deep learning benefits from an emerging class of AI accelerators. Some of these accelerators' designs are general enough for compute-intensive applications beyond AI and Cloud TPU is one such example. In this paper, we demonstrate a novel approach using TensorFlow on Cloud TPU to simulate the two-dimensional Ising Model. TensorFlow and Cloud TPU framework enable the simple and readable code to express the complicated distributed algorithm without compromising the performance. Our code implementation fits into a small Jupyter Notebook and fully utilizes Cloud TPU's efficient matrix operation and dedicated high speed inter-chip connection. The performance is highly competitive: it outperforms the best published benchmarks to our knowledge by 60% in single-core and 250% in multi-core with good linear scaling. When compared to Tesla V100 GPU, the single-core performance maintains a ~10% gain. We also demonstrate that using low precision arithmetic---bfloat16---does not compromise the correctness of the simulation results.

cs.DC↗

arXiv2019

Global stability for nonlinear wave equations with multi-localized initial data

In this paper, we initiate the study of the global stability of nonlinear wave equations with initial data that are not required to be localized around a single point. More precisely, we allow small initial data localized around any finite collection of points which can be arbitrarily far from one another. Existing techniques do not directly apply to this setting because they require norms with radial weights away from some center to be small. The smallness we require on the data is measured in a norm which does not depend on the scale of the configuration of the data. Our method of proof relies on a close analysis of the geometry of the interaction between waves originating from different sources. We prove estimates on the bilinear forms encoding the interaction, which allow us to show improved bounds for the energy of the solution. We finally apply a variant of the vector field method involving modified Klainerman--Sobolev estimates to prove global stability. As a corollary of our proof, we are able to show global existence for a class of data whose $H^1$ norm is arbitrarily large.

math.AP↗

arXiv2019

Machine Learning in High Energy Physics Community White Paper

Machine learning has been applied to several problems in particle physics research, beginning with applications to high-level physics analysis in the 1990s and 2000s, followed by an explosion of applications in particle and event identification and reconstruction in the 2010s. In this document we discuss promising future research and development areas for machine learning in particle physics. We detail a roadmap for their implementation, software and hardware resource requirements, collaborative initiatives with the data science community, academia and industry, and training the particle physics community in data science. The main objective of the document is to connect and motivate these areas of research and development with the physics drivers of the High-Luminosity Large Hadron Collider and future neutrino experiments and identify the resource needs for their implementation. Additionally we identify areas where collaboration with external communities will be of great benefit.

physics.comp-ph↗

arXiv2019

Scaling Up Collaborative Filtering Data Sets through Randomized Fractal Expansions

Recommender system research suffers from a disconnect between the size of academic data sets and the scale of industrial production systems. In order to bridge that gap, we propose to generate large-scale user/item interaction data sets by expanding pre-existing public data sets. Our key contribution is a technique that expands user/item incidence matrices matrices to large numbers of rows (users), columns (items), and non-zero values (interactions). The proposed method adapts Kronecker Graph Theory to preserve key higher order statistical properties such as the fat-tailed distribution of user engagements, item popularity, and singular value spectra of user/item interaction matrices. Preserving such properties is key to building large realistic synthetic data sets which in turn can be employed reliably to benchmark recommender systems and the systems employed to train them. We further apply our stochastic expansion algorithm to the binarized MovieLens 20M data set, which comprises 20M interactions between 27K movies and 138K users. The resulting expanded data set has 1.2B ratings, 2.2M users, and 855K items, which can be scaled up or down.

cs.LG↗

arXiv2019

Scalable Realistic Recommendation Datasets through Fractal Expansions

Recommender System research suffers currently from a disconnect between the size of academic data sets and the scale of industrial production systems. In order to bridge that gap we propose to generate more massive user/item interaction data sets by expanding pre-existing public data sets. User/item incidence matrices record interactions between users and items on a given platform as a large sparse matrix whose rows correspond to users and whose columns correspond to items. Our technique expands such matrices to larger numbers of rows (users), columns (items) and non zero values (interactions) while preserving key higher order statistical properties. We adapt the Kronecker Graph Theory to user/item incidence matrices and show that the corresponding fractal expansions preserve the fat-tailed distributions of user engagements, item popularity and singular value spectra of user/item interaction matrices. Preserving such properties is key to building large realistic synthetic data sets which in turn can be employed reliably to benchmark Recommender Systems and the systems employed to train them. We provide algorithms to produce such expansions and apply them to the MovieLens 20 million data set comprising 20 million ratings of 27K movies by 138K users. The resulting expanded data set has 10 billion ratings, 864K items and 2 million users in its smaller version and can be scaled up or down. A larger version features 655 billion ratings, 7 million items and 17 million users.

cs.IR↗

arXiv2018

Efficient Training on Very Large Corpora via Gramian Estimation

We study the problem of learning similarity functions over very large corpora using neural network embedding models. These models are typically trained using SGD with sampling of random observed and unobserved pairs, with a number of samples that grows quadratically with the corpus size, making it expensive to scale to very large corpora. We propose new efficient methods to train these models without having to sample unobserved pairs. Inspired by matrix factorization, our approach relies on adding a global quadratic penalty to all pairs of examples and expressing this term as the matrix-inner-product of two generalized Gramians. We show that the gradient of this term can be efficiently computed by maintaining estimates of the Gramians, and develop variance reduction schemes to improve the quality of the estimates. We conduct large-scale experiments that show a significant improvement in training time and generalization quality compared to traditional sampling methods.

stat.ML↗

arXiv2018

An image-based array trigger for Imaging Atmospheric Cherenkov Telescope Arrays

It is anticipated that forthcoming, next generation, atmospheric Cherenkov telescope arrays will include a number of medium-sized telescopes that are constructed using a dual-mirror Schwarzschild-Couder configuration. These telescopes will sample a wide ($8^{\circ}$) field of view using a densely pixelated camera comprising over $10^{4}$ individual readout channels. A readout frequency congruent with the expected single-telescope trigger rates would result in substantial data rates. To ameliorate these data rates, a novel, hardware-level Distributed Intelligent Array Trigger (DIAT) is envisioned. A copy of the DIAT operates autonomously at each telescope and uses reduced resolution imaging data from a limited subset of nearby telescopes to veto events prior to camera readout {and any subsequent network transmission of camera data that is required for centralized storage or aggregation}. We present the results of Monte-Carlo simulations that evaluate the efficacy of a "Parallax width" discriminator that can be used by the DIAT to efficiently distinguish between genuine gamma-ray initiated events and unwanted background events that are initiated by hadronic cosmic rays.

astro-ph.IM↗

arXiv2017

Probing low noise at the MOS interface with a spin-orbit qubit

The silicon metal-oxide-semiconductor (MOS) material system is technologically important for the implementation of electron spin-based quantum information technologies. Researchers predict the need for an integrated platform in order to implement useful computation, and decades of advancements in silicon microelectronics fabrication lends itself to this challenge. However, fundamental concerns have been raised about the MOS interface (e.g. trap noise, variations in electron g-factor and practical implementation of multi-QDs). Furthermore, two-axis control of silicon qubits has, to date, required the integration of non-ideal components (e.g. microwave strip-lines, micro-magnets, triple quantum dots, or introduction of donor atoms). In this paper, we introduce a spin-orbit (SO) driven singlet-triplet (ST) qubit in silicon, demonstrating all-electrical two-axis control that requires no additional integrated elements and exhibits charge noise properties equivalent to other more model, but less commercially mature, semiconductor systems. We demonstrate the ability to tune an intrinsic spin-orbit interface effect, which is consistent with Rashba and Dresselhaus contributions that are remarkably strong for a low spin-orbit material such as silicon. The qubit maintains the advantages of using isotopically enriched silicon for producing a quiet magnetic environment, measuring spin dephasing times of 1.6 $μ$s using 99.95% $^{28}$Si epitaxy for the qubit, comparable to results from other isotopically enhanced silicon ST qubit systems. This work, therefore, demonstrates that the interface inherently provides properties for two-axis control, and the technologically important MOS interface does not add additional detrimental qubit noise.

cond-mat.mes-hall↗

arXiv2012

Design, Installation, and Initial Commissioning of the MTA Beamline

A new experimental area designed to develop, test and verify muon ionization cooling apparatus using the 400-MeV Fermilab Linac proton beam has been fully installed and is presently being commissioned. Initially, this area was used for cryogenic tests of liquid-hydrogen absorbers for the MUCOOL R&D program and, now, for high-power beam tests of absorbers, high-gradient rf cavities in the presence of magnetic fields (including gas-filled cavities), and other prototype muon-cooling apparatus. The experimental scenarios being developed for muon facilities involve collection, capture, and cooling of large-emittance, high-intensity muon beams--~10**13 muons, so that conclusive tests of the apparatus require full Linac beam, which is 1.6 x 10**13 p/pulse. To support the muon cooling facility, this new primary beamline extracts and transports beam directly from the Linac to the test facility. The design concept for the MuCool facility is taken from an earlier proposal, but modifications were necessary to accommodate high-intensity beam, cryogenics, and the increased scale of the cooling experiments. Further, the different mode of operation to provide precision line incorporates a specialized section and utilizes a measurements of Linac beam parameters. This paper reports on the technical details of the MuCool beamline for both modes.

physics.acc-ph↗