arXiv subjects

Chris Watkins

Publications and source records attributed to Chris Watkins.

13 recordsLinked to original sources

arXiv2023

Shades of meaning: Uncovering the geometry of ambiguous word representations through contextualised language models

Lexical ambiguity presents a profound and enduring challenge to the language sciences. Researchers for decades have grappled with the problem of how language users learn, represent and process words with more than one meaning. Our work offers new insight into psychological understanding of lexical ambiguity through a series of simulations that capitalise on recent advances in contextual language models. These models have no grounded understanding of the meanings of words at all; they simply learn to predict words based on the surrounding context provided by other words. Yet, our analyses show that their representations capture fine-grained meaningful distinctions between unambiguous, homonymous, and polysemous words that align with lexicographic classifications and psychological theorising. These findings provide quantitative support for modern psychological conceptualisations of lexical ambiguity and raise new challenges for understanding of the way that contextual information shapes the meanings of words across different timescales.

cs.CL↗

arXiv2023

Improving Expert Specialization in Mixture of Experts

Mixture of experts (MoE), introduced over 20 years ago, is the simplest gated modular neural network architecture. There is renewed interest in MoE because the conditional computation allows only parts of the network to be used during each inference, as was recently demonstrated in large scale natural language processing models. MoE is also of potential interest for continual learning, as experts may be reused for new tasks, and new experts introduced. The gate in the MoE architecture learns task decompositions and individual experts learn simpler functions appropriate to the gate's decomposition. In this paper: (1) we show that the original MoE architecture and its training method do not guarantee intuitive task decompositions and good expert utilization, indeed they can fail spectacularly even for simple data such as MNIST and FashionMNIST; (2) we introduce a novel gating architecture, similar to attention, that improves performance and results in a lower entropy task decomposition; and (3) we introduce a novel data-driven regularization that improves expert specialization. We empirically validate our methods on MNIST, FashionMNIST and CIFAR-100 datasets.

cs.LG↗

arXiv2022

Evolution with recombination as a Metropolis-Hastings sampling procedure

This work presents a population genetic model of evolution, which includes haploid selection, mutation, recombination, and drift. The mutation-selection equilibrium can be expressed exactly in closed form for arbitrary fitness functions without resorting to diffusion approximations. Tractability is achieved by generating new offspring using n-parent rather than 2-parent recombination. While this enforces linkage equilibrium among offspring, it allows analysis of the whole population under linkage disequilibrium. We derive a general and exact relationship between fitness fluctuations and response to selection. Our assumptions allow analytical calculation of the stationary distribution of the model for a variety of non-trivial fitness functions. These results allow us to speak to genetic architecture, i.e., what stationary distributions result from different fitness functions. This paper presents methods for exactly deriving stationary states for finite and infinite populations. This method can be applied to many fitness functions, and we give exact calculations for four of these. These results allow us to investigate metastability, tradeoffs between fitness functions, and even consider error-correcting codes

q-bio.PE↗

arXiv2020

An evolutionary model that satisfies detailed balance

We propose a class of evolutionary models that involves an arbitrary exchangeable process as the breeding process and different selection schemes. In those models, a new genome is born according to the breeding process, and then a genome is removed according to the selection scheme that involves fitness. Thus the population size remains constant. The process evolves according to a Markov chain, and, unlike in many other existing models, the stationary distribution -- so called mutation-selection equilibrium -- can be easily found and studied. The behaviour of the stationary distribution when the population size increases is our main object of interest. Several phase-transition theorems are proved.

cs.NE↗

arXiv2020

Epidemic contact tracing with smartphone sensors

Contact tracing is widely considered as an effective procedure in the fight against epidemic diseases. However, one of the challenges for technology based contact tracing is the high number of false positives, questioning its trust-worthiness and efficiency amongst the wider population for mass adoption. To this end, this paper proposes a novel, yet practical smartphone-based contact tracing approach, employing WiFi and acoustic sound for relative distance estimate, in addition to the air pressure and the magnetic field for ambient environment matching. We present a model combining 6 smartphone sensors, prioritising some of them when certain conditions are met. We empirically verified our approach in various realistic environments to demonstrate an achievement of up to 95% fewer false positives, and 62% more accurate than Bluetooth-only system. To the best of our knowledge, this paper was one of the first work to propose a combination of smartphone sensors for contact tracing.

cs.CY↗

arXiv2020

A Metric Learning Approach to Anomaly Detection in Video Games

With the aim of designing automated tools that assist in the video game quality assurance process, we frame the problem of identifying bugs in video games as an anomaly detection (AD) problem. We develop State-State Siamese Networks (S3N) as an efficient deep metric learning approach to AD in this context and explore how it may be used as part of an automated testing tool. Finally, we show by empirical evaluation on a series of Atari games, that S3N is able to learn a meaningful embedding, and consequently is able to identify various common types of video game bugs.

cs.LG↗

arXiv2020

A review of smartphones based indoor positioning: challenges and applications

The continual proliferation of mobile devices has encouraged much effort in using the smartphones for indoor positioning. This article is dedicated to review the most recent and interesting smartphones based indoor navigation systems, ranging from electromagnetic to inertia to visible light ones, with an emphasis on their unique challenges and potential real-world applications. A taxonomy of smartphones sensors will be introduced, which serves as the basis to categorise different positioning systems for reviewing. A set of criteria to be used for the evaluation purpose will be devised. For each sensor category, the most recent, interesting and practical systems will be examined, with detailed discussion on the open research questions for the academics, and the practicality for the potential clients.

eess.SP↗

arXiv2019

Thermodynamic Computing

The hardware and software foundations laid in the first half of the 20th Century enabled the computing technologies that have transformed the world, but these foundations are now under siege. The current computing paradigm, which is the foundation of much of the current standards of living that we now enjoy, faces fundamental limitations that are evident from several perspectives. In terms of hardware, devices have become so small that we are struggling to eliminate the effects of thermodynamic fluctuations, which are unavoidable at the nanometer scale. In terms of software, our ability to imagine and program effective computational abstractions and implementations are clearly challenged in complex domains. In terms of systems, currently five percent of the power generated in the US is used to run computing systems - this astonishing figure is neither ecologically sustainable nor economically scalable. Economically, the cost of building next-generation semiconductor fabrication plants has soared past $10 billion. All of these difficulties - device scaling, software complexity, adaptability, energy consumption, and fabrication economics - indicate that the current computing paradigm has matured and that continued improvements along this path will be limited. If technological progress is to continue and corresponding social and economic benefits are to continue to accrue, computing must become much more capable, energy efficient, and affordable. We propose that progress in computing can continue under a united, physically grounded, computational paradigm centered on thermodynamics. Herein we propose a research agenda to extend these thermodynamic foundations into complex, non-equilibrium, self-organizing systems and apply them holistically to future computing systems that will harness nature's innate computational capacity. We call this type of computing "Thermodynamic Computing" or TC.

cs.CY↗

arXiv2019

Explaining Deep Learning Models with Constrained Adversarial Examples

Machine learning algorithms generally suffer from a problem of explainability. Given a classification result from a model, it is typically hard to determine what caused the decision to be made, and to give an informative explanation. We explore a new method of generating counterfactual explanations, which instead of explaining why a particular classification was made explain how a different outcome can be achieved. This gives the recipients of the explanation a better way to understand the outcome, and provides an actionable suggestion. We show that the introduced method of Constrained Adversarial Examples (CADEX) can be used in real world applications, and yields explanations which incorporate business or domain constraints such as handling categorical attributes and range constraints.

cs.LG↗

arXiv2017

Co-location Epidemic Tracking on London Public Transports Using Low Power Mobile Magnetometer

The public transports provide an ideal means to enable contagious diseases transmission. This paper introduces a novel idea to detect co-location of people in such environment using just the ubiquitous geomagnetic field sensor on the smart phone. Essentially, given that all passengers must share the same journey between at least two consecutive stations, we have a long window to match the user trajectory. Our idea was assessed over a painstakingly survey of over 150 kilometres of travelling distance, covering different parts of London, using the overground trains, the underground tubes and the buses.

cs.CY↗

arXiv2014

Sex as Gibbs Sampling: a probability model of evolution

We show that evolutionary computation can be implemented as standard Markov-chain Monte-Carlo (MCMC) sampling. With some care, `genetic algorithms' can be constructed that are reversible Markov chains that satisfy detailed balance; it follows that the stationary distribution of populations is a Gibbs distribution in a simple factorised form. For some standard and popular nonparametric probability models, we exhibit Gibbs-sampling procedures that are plausible genetic algorithms. At mutation-selection equilibrium, a population of genomes is analogous to a sample from a Bayesian posterior, and the genomes are analogous to latent variables. We suggest this is a general, tractable, and insightful formulation of evolutionary computation in terms of standard machine learning concepts and techniques. In addition, we show that evolutionary processes in which selection acts by differences in fecundity are not reversible, and also that it is not possible to construct reversible evolutionary models in which each child is produced by only two parents.

q-bio.PE↗

arXiv2013

Intercomparison of General Circulation Models for Hot Extrasolar Planets

We compare five general circulation models (GCMs) which have been recently used to study hot extrasolar planet atmospheres (BOB, CAM, IGCM, MITgcm, and PEQMOD), under three test cases useful for assessing model convergence and accuracy. Such a broad, detailed intercomparison has not been performed thus far for extrasolar planets study. The models considered all solve the traditional primitive equations, but employ different numerical algorithms or grids (e.g., pseudospectral and finite volume, with the latter separately in longitude-latitude and `cubed-sphere' grids). The test cases are chosen to cleanly address specific aspects of the behaviors typically reported in hot extrasolar planet simulations: 1) steady-state, 2) nonlinearly evolving baroclinic wave, and 3) response to fast timescale thermal relaxation. When initialized with a steady jet, all models maintain the steadiness, as they should -- except MITgcm in cubed-sphere grid. A very good agreement is obtained for a baroclinic wave evolving from an initial instability in pseudospectral models (only). However, exact numerical convergence is still not achieved across the pseudospectral models: amplitudes and phases are observably different. When subject to a typical `hot-Jupiter'-like forcing, all five models show quantitatively different behavior -- although qualitatively similar, time-variable, quadrupole-dominated flows are produced. Hence, as have been advocated in several past studies, specific quantitative predictions (such as the location of large vortices and hot regions) by GCMs should be viewed with caution. Overall, in the tests considered here, pseudospectral models in pressure coordinate (PEBOB and PEQMOD) perform the best and MITgcm in cubed-sphere grid performs the worst.

astro-ph.EP↗

arXiv2010

Gravity Waves on Hot Extrasolar Planets: I. Propagation and Interaction with the Background

We study the effects of gravity waves, or g-modes, on hot extrasolar planets. These planets are expected to possess stably-stratified atmospheres, which support gravity waves. In this paper, we review the derivation of the equation that governs the linear dynamics of gravity waves and describe its application to a hot extrasolar planet, using HD209458 b as a generic example. We find that gravity waves can exhibit a wide range of behaviors, even for a single atmospheric profile. The waves can significantly accelerate or decelerate the background mean flow, depending on the difference between the wave phase and mean flow speeds. In addition, the waves can provide significant heating (~100 to ~1000 K per planetary rotation), especially to the region of the atmosphere above about 10 scale heights from the excitation region. Furthermore, by propagating horizontally, gravity waves provide a mechanism for transporting momentum and heat from the dayside of a tidally locked planet to its nightside. We discuss work that needs to be undertaken to incorporate these effects in current atmosphere models of extrasolar planets.

astro-ph.EP↗