arXiv subjects

Omer Gottesman

Publications and source records attributed to Omer Gottesman.

30 records · Page 2Linked to original sources

arXiv2019

A general method for regularizing tensor decomposition methods via pseudo-data

Tensor decomposition methods allow us to learn the parameters of latent variable models through decomposition of low-order moments of data. A significant limitation of these algorithms is that there exists no general method to regularize them, and in the past regularization has mostly been performed using bespoke modifications to the algorithms, tailored for the particular form of the desired regularizer. We present a general method of regularizing tensor decomposition methods which can be used for any likelihood model that is learnable using tensor decomposition methods and any differentiable regularization function by supplementing the training data with pseudo-data. The pseudo-data is optimized to balance two terms: being as close as possible to the true data and enforcing the desired regularization. On synthetic, semi-synthetic and real data, we demonstrate that our method can improve inference accuracy and regularize for a broad range of goals including transfer learning, sparsity, interpretability, and orthogonality of the learned parameters.

stat.ML↗

arXiv2019

Combining Parametric and Nonparametric Models for Off-Policy Evaluation

We consider a model-based approach to perform batch off-policy evaluation in reinforcement learning. Our method takes a mixture-of-experts approach to combine parametric and non-parametric models of the environment such that the final value estimate has the least expected error. We do so by first estimating the local accuracy of each model and then using a planner to select which model to use at every time step as to minimize the return error estimate along entire trajectories. Across a variety of domains, our mixture-based approach outperforms the individual models alone as well as state-of-the-art importance sampling-based estimators.

cs.LG↗

arXiv2019

Representation Balancing MDPs for Off-Policy Policy Evaluation

We study the problem of off-policy policy evaluation (OPPE) in RL. In contrast to prior work, we consider how to estimate both the individual policy value and average policy value accurately. We draw inspiration from recent work in causal reasoning, and propose a new finite sample generalization error bound for value estimates from MDP models. Using this upper bound as an objective, we develop a learning algorithm of an MDP model with a balanced representation, and show that our approach can yield substantially lower MSE in common synthetic benchmarks and a HIV treatment simulation domain.

cs.LG↗

arXiv2019

Improving Sepsis Treatment Strategies by Combining Deep and Kernel-Based Reinforcement Learning

Sepsis is the leading cause of mortality in the ICU. It is challenging to manage because individual patients respond differently to treatment. Thus, tailoring treatment to the individual patient is essential for the best outcomes. In this paper, we take steps toward this goal by applying a mixture-of-experts framework to personalize sepsis treatment. The mixture model selectively alternates between neighbor-based (kernel) and deep reinforcement learning (DRL) experts depending on patient's current history. On a large retrospective cohort, this mixture-based approach outperforms physician, kernel only, and DRL-only experts.

cs.LG↗

arXiv2018

Behaviour Policy Estimation in Off-Policy Policy Evaluation: Calibration Matters

In this work, we consider the problem of estimating a behaviour policy for use in Off-Policy Policy Evaluation (OPE) when the true behaviour policy is unknown. Via a series of empirical studies, we demonstrate how accurate OPE is strongly dependent on the calibration of estimated behaviour policy models: how precisely the behaviour policy is estimated from data. We show how powerful parametric models such as neural networks can result in highly uncalibrated behaviour policy models on a real-world medical dataset, and illustrate how a simple, non-parametric, k-nearest neighbours model produces better calibrated behaviour policy estimates and can be used to obtain superior importance sampling-based OPE estimates.

cs.LG↗

arXiv2018

A state variable for crumpled thin sheets

Despite the apparent ease with which a sheet of paper is crumpled and tossed away, crumpling dynamics are often considered a paradigm of complexity. This complexity arises from the infinite number of configurations a disordered crumpled sheet can take. Here we experimentally show that key aspects of crumpling have a very simple description; the evolution of the damage in crumpling dynamics can largely be described by a single global quantity, the total length of all creases. We follow the evolution of the damage network in repetitively crumpled elastoplastic sheets, and show that the dynamics of this quantity are deterministic, and depend only on the instantaneous state of the crease network and not at all on the crumpling history. We also show that this global quantity captures the crumpling dynamics of a sheet crumpled for the first time. This leads to a remarkable reduction in complexity, allowing a description of a highly disordered system by a single state parameter. Similar strategies may also be useful in analyzing other systems that evolve under geometric and mechanical constraints, from faulting of tectonic plates to the evolution of proteins.

cond-mat.soft↗

arXiv2018

Evaluating Reinforcement Learning Algorithms in Observational Health Settings

Much attention has been devoted recently to the development of machine learning algorithms with the goal of improving treatment policies in healthcare. Reinforcement learning (RL) is a sub-field within machine learning that is concerned with learning how to make sequences of decisions so as to optimize long-term effects. Already, RL algorithms have been proposed to identify decision-making strategies for mechanical ventilation, sepsis management and treatment of schizophrenia. However, before implementing treatment policies learned by black-box algorithms in high-stakes clinical decision problems, special care must be taken in the evaluation of these policies. In this document, our goal is to expose some of the subtleties associated with evaluating RL algorithms in healthcare. We aim to provide a conceptual starting point for clinical and computational researchers to ask the right questions when designing and evaluating algorithms for new ways of treating patients. In the following, we describe how choices about how to summarize a history, variance of statistical estimators, and confounders in more ad-hoc measures can result in unreliable, even misleading estimates of the quality of a treatment policy. We also provide suggestions for mitigating these effects---for while there is much promise for mining observational health data to uncover better treatment policies, evaluation must be performed thoughtfully.

cs.LG↗

arXiv2017

Weighted Tensor Decomposition for Learning Latent Variables with Partial Data

Tensor decomposition methods are popular tools for learning latent variables given only lower-order moments of the data. However, the standard assumption is that we have sufficient data to estimate these moments to high accuracy. In this work, we consider the case in which certain dimensions of the data are not always observed---common in applied settings, where not all measurements may be taken for all observations---resulting in moment estimates of varying quality. We derive a weighted tensor decomposition approach that is computationally as efficient as the non-weighted approach, and demonstrate that it outperforms methods that do not appropriately leverage these less-observed dimensions.

stat.ML↗

arXiv2017

Nonmonotonic Aging and Memory Retention in Disordered Mechanical Systems

We observe non-monotonic aging and memory effects, two hallmarks of glassy dynamics, in two disordered mechanical systems: crumpled thin sheets and elastic foams. Under fixed compression, both systems exhibit monotonic non-exponential relaxation. However, when after a certain waiting time the compression is partially reduced, both systems exhibit a non-monotonic response: the normal force first increases over many minutes or even hours until reaching a peak value, and only then relaxation is resumed. The peak-time scales linearly with the waiting time, indicating that these systems retain long-lasting memory of previous conditions. Our results and the measured scaling relations are in good agreement with a theoretical model recently used to describe observations of monotonic aging in several glassy systems, suggesting that the non-monotonic behavior may be generic and that a-thermal systems can show genuine glassy behavior.

cond-mat.soft↗

arXiv2016

On the incompressibility of cylindrical origami patterns

The art and science of folding intricate three-dimensional structures out of paper has occupied artists, designers, engineers, and mathematicians for decades, culminating in the design of deployable structures and mechanical metamaterials. Here we investigate the axial compressibility of origami cylinders, i.e., cylindrical structures folded from rectangular sheets of paper. We prove, using geometric arguments, that a general fold pattern only allows for a finite number of \emph{isometric} cylindrical embeddings. Therefore, compressibility of such structures requires either stretching the material or deforming the folds. Our result considerably restricts the space of constructions that must be searched when designing new types of origami-based rigid-foldable deployable structures and metamaterials.

physics.comp-ph↗

arXiv2013

Furrows in the wake of propagating d-cones

We investigate the formation dynamics of plastic creases in thin elasto-plastic sheets. In contrast to the commonly accepted description of crumpled thin sheets, which asserts that creases form only by elastic interaction between two d-cones, the creases we study in this letter are created by plastic deformations left in the wake of a single propagating d-cone. Upon application of load, a d-cone initially remains stationary and responds by deforming globally. However, above a critical load, the d-cone undergoes a sharpening transition that focuses the stresses at its tip, allowing it to propagate along the sheet, leaving a furrow-like scar in its wake. Our results show that the dynamics of plastic defect creation are important for predicting the final geometry and statistics of a defect network in a crumpled thin sheet.

cond-mat.soft↗

arXiv2012

Multiple extinction routes in stochastic population models

Isolated populations ultimately go extinct because of the intrinsic noise of elementary processes. In multi-population systems extinction of a population may occur via more than one route. We investigate this generic situation in a simple predator-prey (or infected-susceptible) model. The predator and prey populations may coexist for a long time but ultimately both go extinct. In the first extinction route the predators go extinct first, whereas the prey thrive for a long time and then also go extinct. In the second route the prey go extinct first causing a rapid extinction of the predators. Assuming large sub-population sizes in the coexistence state, we compare the probabilities of each of the two extinction routes and predict the most likely path of the sub-populations to extinction. We also suggest an effective three-state master equation for the probabilities to observe the coexistence state, the predator-free state and the empty state.

q-bio.PE↗