arXiv subjects

Will Williams

Publications and source records attributed to Will Williams.

5 recordsLinked to original sources

arXiv2020

Hierarchical Quantized Autoencoders

Despite progress in training neural networks for lossy image compression, current approaches fail to maintain both perceptual quality and abstract features at very low bitrates. Encouraged by recent success in learning discrete representations with Vector Quantized Variational Autoencoders (VQ-VAEs), we motivate the use of a hierarchy of VQ-VAEs to attain high factors of compression. We show that the combination of stochastic quantization and hierarchical latent structure aids likelihood-based image compression. This leads us to introduce a novel objective for training hierarchical VQ-VAEs. Our resulting scheme produces a Markovian series of latent variables that reconstruct images of high-perceptual quality which retain semantically meaningful features. We provide qualitative and quantitative evaluations on the CelebA and MNIST datasets.

cs.LG↗

arXiv2019

Texture Bias Of CNNs Limits Few-Shot Classification Performance

Accurate image classification given small amounts of labelled data (few-shot classification) remains an open problem in computer vision. In this work we examine how the known texture bias of Convolutional Neural Networks (CNNs) affects few-shot classification performance. Although texture bias can help in standard image classification, in this work we show it significantly harms few-shot classification performance. After correcting this bias we demonstrate state-of-the-art performance on the competitive miniImageNet task using a method far simpler than the current best performing few-shot learning approaches.

cs.LG↗

arXiv2018

Discriminative training of RNNLMs with the average word error criterion

In automatic speech recognition (ASR), recurrent neural language models (RNNLM) are typically used to refine hypotheses in the form of lattices or n-best lists, which are generated by a beam search decoder with a weaker language model. The RNNLMs are usually trained generatively using the perplexity (PPL) criterion on large corpora of grammatically correct text. However, the hypotheses are noisy, and the RNNLM doesn't always make the choices that minimise the metric we optimise for, the word error rate (WER). To address this mismatch we propose to use a task specific loss to train an RNNLM to discriminate between multiple hypotheses within lattice rescoring scenario. By fine-tuning the RNNLM on lattices with the average edit distance loss, we show that we obtain a 1.9% relative improvement in word error rate over a purely generatively trained model.

cs.CL↗

arXiv2015

Scaling Recurrent Neural Network Language Models

This paper investigates the scaling properties of Recurrent Neural Network Language Models (RNNLMs). We discuss how to train very large RNNs on GPUs and address the questions of how RNNLMs scale with respect to model size, training-set size, computational costs and memory. Our analysis shows that despite being more costly to train, RNNLMs obtain much lower perplexities on standard benchmarks than n-gram models. We train the largest known RNNs and present relative word error rates gains of 18% on an ASR task. We also present the new lowest perplexities on the recently released billion word language modelling benchmark, 1 BLEU point gain on machine translation and a 17% relative hit rate gain in word prediction.

cs.CL↗

arXiv2007

Compression and localization of an atomic cloud in a time dependent optical lattice

We analyze a method of compressing a cloud of cold atoms by dynamic control of a far off resonance optical lattice. We show that by reducing the lattice spacing either continuously or in discrete steps while cooling the atoms with optical molasses large compression factors can be achieved. Particle motion in the time dependent lattice is studied numerically using a three dimensional semiclassical model. Two experimentally realistic models are analyzed. In the first we continuously vary the lattice beam angles to compress atoms initially in a Gaussian distributed cloud with standard deviation of $250 μ\rm m$ into a single site of a two-dimensional lattice of area $A\sim 35\times 35 λ^2$, with $λ$ the wavelength of the lattice beams. This results in an optical depth for an on-resonant probe beam $>80$ which is an increase by a factor of about 1800 compared to the uncompressed cloud. In the second approach we use a discrete set of lattice beam angles to decrease the spatial scale of the cloud by a factor of 500, and localize a few atoms to a single lattice site with an area $A\stackrel{<}{\sim} λ^2.$

physics.atom-ph↗