arXiv subjects

James Harrison

Publications and source records attributed to James Harrison.

50 records · Page 3Linked to original sources

arXiv2020

Sparse Longitudinal Representations of Electronic Health Record Data for the Early Detection of Chronic Kidney Disease in Diabetic Patients

Chronic kidney disease (CKD) is a gradual loss of renal function over time, and it increases the risk of mortality, decreased quality of life, as well as serious complications. The prevalence of CKD has been increasing in the last couple of decades, which is partly due to the increased prevalence of diabetes and hypertension. To accurately detect CKD in diabetic patients, we propose a novel framework to learn sparse longitudinal representations of patients' medical records. The proposed method is also compared with widely used baselines such as Aggregated Frequency Vector and Bag-of-Pattern in Sequences on real EHR data, and the experimental results indicate that the proposed model achieves higher predictive performance. Additionally, the learned representations are interpreted and visualized to bring clinical insights.

cs.LG↗

arXiv2020

Continuous Meta-Learning without Tasks

Meta-learning is a promising strategy for learning to efficiently learn within new tasks, using data gathered from a distribution of tasks. However, the meta-learning literature thus far has focused on the task segmented setting, where at train-time, offline data is assumed to be split according to the underlying task, and at test-time, the algorithms are optimized to learn in a single task. In this work, we enable the application of generic meta-learning algorithms to settings where this task segmentation is unavailable, such as continual online learning with a time-varying task. We present meta-learning via online changepoint analysis (MOCA), an approach which augments a meta-learning algorithm with a differentiable Bayesian changepoint detection scheme. The framework allows both training and testing directly on time series data without segmenting it into discrete tasks. We demonstrate the utility of this approach on a nonlinear meta-regression benchmark as well as two meta-image-classification benchmarks.

cs.LG↗

arXiv2020

Deep Reinforcement Learning amidst Lifelong Non-Stationarity

As humans, our goals and our environment are persistently changing throughout our lifetime based on our experiences, actions, and internal and external drives. In contrast, typical reinforcement learning problem set-ups consider decision processes that are stationary across episodes. Can we develop reinforcement learning algorithms that can cope with the persistent change in the former, more realistic problem settings? While on-policy algorithms such as policy gradients in principle can be extended to non-stationary settings, the same cannot be said for more efficient off-policy algorithms that replay past experiences when learning. In this work, we formalize this problem setting, and draw upon ideas from the online learning and probabilistic inference literature to derive an off-policy RL algorithm that can reason about and tackle such lifelong non-stationarity. Our method leverages latent variable models to learn a representation of the environment from current and past experiences, and performs off-policy RL with this representation. We further introduce several simulation environments that exhibit lifelong non-stationarity, and empirically find that our approach substantially outperforms approaches that do not reason about environment shift.

cs.LG↗

arXiv2019

Electromagnetic finite-size effects to the hadronic vacuum polarisation

In order to reach (sub-)per cent level precision in lattice calculations of the hadronic vacuum polarisation, isospin breaking corrections must be included. This requires introducing QED on the lattice, and the associated finite-size effects are potentially large due to the absence of a mass gap. This means that the finite-size effects scale as an inverse polynomial in $L$ rather than being exponentially suppressed. Considering the $\mathcal{O}(α)$ corrected hadronic vacuum polarisation in QED$_{\mathrm{L}}$ with scalar QED as an effective theory, we show that the first possible term, which is of order $1/L^{2}$, vanishes identically so that the finite-size effects start at order $1/L^{3}$. This cancellation is understood from the neutrality of the currents involved, and we show that this cancellation is universal by also including form factors for the pions. We find good numerical agreement with lattice perturbation theory calculations, as well as, up to exponentially suppressed terms, scalar QED lattice simulations.

hep-lat↗

arXiv2019

An improved understanding of the roles of atomic processes and power balance in divertor target ion current loss during detachment

The process of divertor detachment, whereby heat and particle fluxes to divertor surfaces are strongly diminished, is required to reduce heat loading and erosion in a magnetic fusion reactor to acceptable levels. In this paper the physics leading to the decrease of the total divertor ion current (It), or 'roll-over', is experimentally explored on the TCV tokamak through characterization of the location, magnitude and role of the various divertor ion sinks and sources including a complete analysis of particle and power balance. These first measurements of the profiles of divertor ionisation and hydrogenic radiation along the divertor leg are enabled through novel spectroscopic techniques. Over a range in TCV plasma conditions (plasma current and electron density, with/without impurity-seeding) the $I_t$ roll-over is ascribed to a drop in the divertor ion source; recombination remains small or negligible farther into the detachment process. The ion source reduction is driven by both a reduction in the power available for ionization, Precl, and concurrent increase in the energy required per ionisation, $E_{ion}$: often described as 'power starvation' (or 'power limitation'). The detachment threshold is found experimentally (in agreement with analytic model predictions) to be $\sim P_{recl}/I_t {E_{ion}} \sim 2$, corresponding to a target electron temperature, $T_t \sim E_{ion}/γ$ where $γ$ is the sheath transmission coefficient. The target pressure reduction, required to reduce the target ion current, is driven both by volumetric momentum loss as well as upstream pressure loss. The measured evolution through detachment of the divertor profile of various ion sources/sinks as well as power losses are quantitatively reproduced through full 2D SOLPS modelling through the detachment process as the core density is varied.

physics.plasm-ph↗

arXiv2019

Theoretical aspects of quantum electrodynamics in a finite volume with periodic boundary conditions

First-principles studies of strongly-interacting hadronic systems using lattice quantum chromodynamics (QCD) have been complemented in recent years with the inclusion of quantum electrodynamics (QED). The aim is to confront experimental results with more precise theoretical determinations, e.g. for the anomalous magnetic moment of the muon and the CP-violating parameters in the decay of mesons. Quantifying the effects arising from enclosing QED in a finite volume remains a primary target of investigations. To this end, finite-volume corrections to hadron masses in the presence of QED have been carefully studied in recent years. This paper extends such studies to the self-energy of moving charged hadrons, both on and away from their mass shell. In particular, we present analytical results for leading finite-volume corrections to the self-energy of spin-0 and spin-$\frac{1}{2}$ particles in the presence of QED on a periodic hypercubic lattice, once the spatial zero mode of the photon is removed, a framework that is called $\mathrm{QED}_{\mathrm{L}}$. By altering modes beyond the zero mode, an improvement scheme is introduced to eliminate the leading finite-volume corrections to masses, with potential applications to other hadronic quantities. Our analytical results are verified by a dedicated numerical study of a lattice scalar field theory coupled to $\mathrm{QED}_{\mathrm{L}}$. Further, this paper offers new perspectives on the subtleties involved in applying low-energy effective field theories in the presence of $\mathrm{QED}_{\mathrm{L}}$, a theory that is rendered non-local with the exclusion of the spatial zero mode of the photon, clarifying recent discussions on this matter.

hep-lat↗

arXiv2019

Learning Sampling Distributions for Robot Motion Planning

A defining feature of sampling-based motion planning is the reliance on an implicit representation of the state space, which is enabled by a set of probing samples. Traditionally, these samples are drawn either probabilistically or deterministically to uniformly cover the state space. Yet, the motion of many robotic systems is often restricted to "small" regions of the state space, due to, for example, differential constraints or collision-avoidance constraints. To accelerate the planning process, it is thus desirable to devise non-uniform sampling strategies that favor sampling in those regions where an optimal solution might lie. This paper proposes a methodology for non-uniform sampling, whereby a sampling distribution is learned from demonstrations, and then used to bias sampling. The sampling distribution is computed through a conditional variational autoencoder, allowing sample generation from the latent space conditioned on the specific planning problem. This methodology is general, can be used in combination with any sampling-based planner, and can effectively exploit the underlying structure of a planning problem while maintaining the theoretical guarantees of sampling-based approaches. Specifically, on several planning problems, the proposed methodology is shown to effectively learn representations for the relevant regions of the state space, resulting in an order of magnitude improvement in terms of success rate and convergence to the optimal cost.

cs.RO↗

arXiv2019

Network Offloading Policies for Cloud Robotics: a Learning-based Approach

Today's robotic systems are increasingly turning to computationally expensive models such as deep neural networks (DNNs) for tasks like localization, perception, planning, and object detection. However, resource-constrained robots, like low-power drones, often have insufficient on-board compute resources or power reserves to scalably run the most accurate, state-of-the art neural network compute models. Cloud robotics allows mobile robots the benefit of offloading compute to centralized servers if they are uncertain locally or want to run more accurate, compute-intensive models. However, cloud robotics comes with a key, often understated cost: communicating with the cloud over congested wireless networks may result in latency or loss of data. In fact, sending high data-rate video or LIDAR from multiple robots over congested networks can lead to prohibitive delay for real-time applications, which we measure experimentally. In this paper, we formulate a novel Robot Offloading Problem --- how and when should robots offload sensing tasks, especially if they are uncertain, to improve accuracy while minimizing the cost of cloud communication? We formulate offloading as a sequential decision making problem for robots, and propose a solution using deep reinforcement learning. In both simulations and hardware experiments using state-of-the art vision DNNs, our offloading strategy improves vision task performance by between 1.3-2.6x of benchmark offloading strategies, allowing robots the potential to significantly transcend their on-board sensing accuracy but with limited cost of cloud communication.

cs.RO↗

arXiv2019

Robust and Adaptive Planning under Model Uncertainty

Planning under model uncertainty is a fundamental problem across many applications of decision making and learning. In this paper, we propose the Robust Adaptive Monte Carlo Planning (RAMCP) algorithm, which allows computation of risk-sensitive Bayes-adaptive policies that optimally trade off exploration, exploitation, and robustness. RAMCP formulates the risk-sensitive planning problem as a two-player zero-sum game, in which an adversary perturbs the agent's belief over the models. We introduce two versions of the RAMCP algorithm. The first, RAMCP-F, converges to an optimal risk-sensitive policy without having to rebuild the search tree as the underlying belief over models is perturbed. The second version, RAMCP-I, improves computational efficiency at the cost of losing theoretical guarantees, but is shown to yield empirical results comparable to RAMCP-F. RAMCP is demonstrated on an n-pull multi-armed bandit problem, as well as a patient treatment scenario.

cs.AI↗

arXiv2018

Meta-Learning Priors for Efficient Online Bayesian Regression

Gaussian Process (GP) regression has seen widespread use in robotics due to its generality, simplicity of use, and the utility of Bayesian predictions. The predominant implementation of GP regression is a nonparameteric kernel-based approach, as it enables fitting of arbitrary nonlinear functions. However, this approach suffers from two main drawbacks: (1) it is computationally inefficient, as computation scales poorly with the number of samples; and (2) it can be data inefficient, as encoding prior knowledge that can aid the model through the choice of kernel and associated hyperparameters is often challenging and unintuitive. In this work, we propose ALPaCA, an algorithm for efficient Bayesian regression which addresses these issues. ALPaCA uses a dataset of sample functions to learn a domain-specific, finite-dimensional feature encoding, as well as a prior over the associated weights, such that Bayesian linear regression in this feature space yields accurate online predictions of the posterior predictive density. These features are neural networks, which are trained via a meta-learning (or "learning-to-learn") approach. ALPaCA extracts all prior information directly from the dataset, rather than restricting prior information to the choice of kernel hyperparameters. Furthermore, by operating in the weight space, it substantially reduces sample complexity. We investigate the performance of ALPaCA on two simple regression problems, two simulated robotic systems, and on a lane-change driving task performed by humans. We find our approach outperforms kernel-based GP regression, as well as state of the art meta-learning approaches, thereby providing a promising plug-in tool for many regression tasks in robotics where scalability and data-efficiency are important.

cs.RO↗

arXiv2018

BaRC: Backward Reachability Curriculum for Robotic Reinforcement Learning

Model-free Reinforcement Learning (RL) offers an attractive approach to learn control policies for high-dimensional systems, but its relatively poor sample complexity often forces training in simulated environments. Even in simulation, goal-directed tasks whose natural reward function is sparse remain intractable for state-of-the-art model-free algorithms for continuous control. The bottleneck in these tasks is the prohibitive amount of exploration required to obtain a learning signal from the initial state of the system. In this work, we leverage physical priors in the form of an approximate system dynamics model to design a curriculum scheme for a model-free policy optimization algorithm. Our Backward Reachability Curriculum (BaRC) begins policy training from states that require a small number of actions to accomplish the task, and expands the initial state distribution backwards in a dynamically-consistent manner once the policy optimization algorithm demonstrates sufficient performance. BaRC is general, in that it can accelerate training of any model-free RL algorithm on a broad class of goal-directed continuous control MDPs. Its curriculum strategy is physically intuitive, easy-to-tune, and allows incorporating physical priors to accelerate training without hindering the performance, flexibility, and applicability of the model-free RL algorithm. We evaluate our approach on two representative dynamic robotic learning problems and find substantial performance improvement relative to previous curriculum generation techniques and naive exploration strategies.

cs.RO↗

arXiv2017

ADAPT: Zero-Shot Adaptive Policy Transfer for Stochastic Dynamical Systems

Model-free policy learning has enabled robust performance of complex tasks with relatively simple algorithms. However, this simplicity comes at the cost of requiring an Oracle and arguably very poor sample complexity. This renders such methods unsuitable for physical systems. Variants of model-based methods address this problem through the use of simulators, however, this gives rise to the problem of policy transfer from simulated to the physical system. Model mismatch due to systematic parameter shift and unmodelled dynamics error may cause sub-optimal or unsafe behavior upon direct transfer. We introduce the Adaptive Policy Transfer for Stochastic Dynamics (ADAPT) algorithm that achieves provably safe and robust, dynamically-feasible zero-shot transfer of RL-policies to new domains with dynamics error. ADAPT combines the strengths of offline policy learning in a black-box source simulator with online tube-based MPC to attenuate bounded model mismatch between the source and target dynamics. ADAPT allows online transfer of policy, trained solely in a simulation offline, to a family of unknown targets without fine-tuning. We also formally show that (i) ADAPT guarantees state and control safety through state-action tubes under the assumption of Lipschitz continuity of the divergence in dynamics and, (ii) ADAPT results in a bounded loss of reward accumulation relative to a policy trained and evaluated in the source environment. We evaluate ADAPT on 2 continuous, non-holonomic simulated dynamical systems with 4 different disturbance models, and find that ADAPT performs between 50%-300% better on mean reward accrual than direct policy transfer.

cs.RO↗

arXiv2017

Isospin Breaking Corrections to the HVP with Domain Wall Fermions

We present results for the QED and strong isospin breaking corrections to the hadronic vacuum polarization using $N_f=2+1$ Domain Wall fermions. QED is included in an electro-quenched setup using two different methods, a stochastic and a perturbative approach. Results and statistical errors from both methods are directly compared with each other.

hep-lat↗

arXiv2016

Electromagnetic Corrections to Meson Masses and the HVP

We present an exploratory study of the electromagnetic corrections to meson masses and the hadronic vacuum polarization using $N_f=2+1$ Domain Wall fermions. These corrections are estimated with two different approaches, a stochastic approach using $U(1)$ gauge configurations for the photon fields, and a perturbative approach through a QED perturbative expansion of the QCD+QED path integral. We compare results and statistical errors from both methods.

hep-lat↗