arXiv subjects

Daniel Lee

Publications and source records attributed to Daniel Lee.

64 records · Page 4Linked to original sources

arXiv2019

Q-Learning for Continuous Actions with Cross-Entropy Guided Policies

Off-Policy reinforcement learning (RL) is an important class of methods for many problem domains, such as robotics, where the cost of collecting data is high and on-policy methods are consequently intractable. Standard methods for applying Q-learning to continuous-valued action domains involve iteratively sampling the Q-function to find a good action (e.g. via hill-climbing), or by learning a policy network at the same time as the Q-function (e.g. DDPG). Both approaches make tradeoffs between stability, speed, and accuracy. We propose a novel approach, called Cross-Entropy Guided Policies, or CGP, that draws inspiration from both classes of techniques. CGP aims to combine the stability and performance of iterative sampling policies with the low computational cost of a policy network. Our approach trains the Q-function using iterative sampling with the Cross-Entropy Method (CEM), while training a policy network to imitate CEM's sampling behavior. We demonstrate that our method is more stable to train than state of the art policy network methods, while preserving equivalent inference time compute costs, and achieving competitive total reward on standard benchmarks.

cs.AI↗

arXiv2019

Newswire versus Social Media for Disaster Response and Recovery

In a disaster situation, first responders need to quickly acquire situational awareness and prioritize response based on the need, resources available and impact. Can they do this based on digital media such as Twitter alone, or newswire alone, or some combination of the two? We examine this question in the context of the 2015 Nepal Earthquakes. Because newswire articles are longer, effective summaries can be helpful in saving time yet giving key content. We evaluate the effectiveness of several unsupervised summarization techniques in capturing key content. We propose a method to link tweets written by the public and newswire articles, so that we can compare their key characteristics: timeliness, whether tweets appear earlier than their corresponding news articles, and content. A novel idea is to view relevant tweets as a summary of the matching news article and evaluate these summaries. Whenever possible, we present both quantitative and qualitative evaluations. One of our main findings is that tweets and newswire articles provide complementary perspectives that form a holistic view of the disaster situation.

cs.IR↗

arXiv2019

Pixels to Plans: Learning Non-Prehensile Manipulation by Imitating a Planner

We present a novel method enabling robots to quickly learn to manipulate objects by leveraging a motion planner to generate "expert" training trajectories from a small amount of human-labeled data. In contrast to the traditional sense-plan-act cycle, we propose a deep learning architecture and training regimen called PtPNet that can estimate effective end-effector trajectories for manipulation directly from a single RGB-D image of an object. Additionally, we present a data collection and augmentation pipeline that enables the automatic generation of large numbers (millions) of training image and trajectory examples with almost no human labeling effort. We demonstrate our approach in a non-prehensile tool-based manipulation task, specifically picking up shoes with a hook. In hardware experiments, PtPNet generates motion plans (open-loop trajectories) that reliably (89% success over 189 trials) pick up four very different shoes from a range of positions and orientations, and reliably picks up a shoe it has never seen before. Compared with a traditional sense-plan-act paradigm, our system has the advantages of operating on sparse information (single RGB-D frame), producing high-quality trajectories much faster than the "expert" planner (300ms versus several seconds), and generalizing effectively to previously unseen shoes.

cs.RO↗

arXiv2018

Eigenvector continuation with subspace learning

A common challenge faced in quantum physics is finding the extremal eigenvalues and eigenvectors of a Hamiltonian matrix in a vector space so large that linear algebra operations on general vectors are not possible. There are numerous efficient methods developed for this task, but they generally fail when some control parameter in the Hamiltonian matrix exceeds some threshold value. In this work we present a new technique called eigenvector continuation that can extend the reach of these methods. The key insight is that while an eigenvector resides in a linear space with enormous dimensions, the eigenvector trajectory generated by smooth changes of the Hamiltonian matrix is well approximated by a very low-dimensional manifold. We prove this statement using analytic function theory and propose an algorithm to solve for the extremal eigenvectors. We benchmark the method using several examples from quantum many-body theory.

nucl-th↗

arXiv2017

Infrared Photometric Properties of 709 Candidate Stellar Bowshock Nebulae

Arcuate infrared nebulae are ubiquitous throughout the Galactic Plane and are candidates for partial shells, bubbles, or bowshocks produced by massive runaway stars. We tabulate infrared photometry for 709 such objects using images from the Spitzer Space Telescope (SST), Wide-Field Infrared Explorer (WISE), and Herschel Space Observatory (HSO). Of the 709 objects identified at 24 or 22 microns, 422 are detected at the HSO 70 micron bandpass. Of these, only 39 are detected at HSO 160 microns. The 70 micron peak surface brightnesses are 0.5 to 2.5 Jy/square arcminute. Color temperatures calculated from the 24 micron to 70 micron ratios range from 80 K to 400 K. Color temperatures from 70 micron to 160 micron ratios are systematically lower, 40 K to 200 K. Both of these temperature are, on average, 75% higher than the nominal temperatures derived by assuming that dust is in steady-state radiative equilibrium. This may be evidence of stellar wind bowshocks sweeping up and heating --- possibly fragmenting but not destroying --- interstellar dust. Infrared luminosity correlates with standoff distance, R_0, as predicted by published hydrodynamical models. Infrared spectral energy distributions are consistent with interstellar dust exposed to a either single radiant energy density, U=10^3 to 10^5 (in more than half of the objects) or a range of radiant energy densities U_min=25 to U_max=10^3 to 10^5 times the mean interstellar value for the remainder. Hence, the central OB stars dominate the energetics, making these enticing laboratories for testing dust models in constrained radiation environments. SEDs are consistent with PAH fractions q_PAH <1% in most objects.

astro-ph.SR↗

arXiv2017

Identifying Reference Spans: Topic Modeling and Word Embeddings help IR

The CL-SciSumm 2016 shared task introduced an interesting problem: given a document D and a piece of text that cites D, how do we identify the text spans of D being referenced by the piece of text? The shared task provided the first annotated dataset for studying this problem. We present an analysis of our continued work in improving our system's performance on this task. We demonstrate how topic models and word embeddings can be used to surpass the previously best performing system.

cs.CL↗

arXiv2017

Extractive Summarization: Limits, Compression, Generalized Model and Heuristics

Due to its promise to alleviate information overload, text summarization has attracted the attention of many researchers. However, it has remained a serious challenge. Here, we first prove empirical limits on the recall (and F1-scores) of extractive summarizers on the DUC datasets under ROUGE evaluation for both the single-document and multi-document summarization tasks. Next we define the concept of compressibility of a document and present a new model of summarization, which generalizes existing models in the literature and integrates several dimensions of the summarization, viz., abstractive versus extractive, single versus multi-document, and syntactic versus semantic. Finally, we examine some new and existing single-document summarization algorithms in a single framework and compare with state of the art summarizers on DUC data.

cs.CL↗

arXiv2016

Toward a Science of Autonomy for Physical Systems: Transportation

Transportation systems are currently being transformed by advances in information and communication technologies. The development of autonomous transportation holds the promise of providing revolutionary improvements in speed, efficiency, safety and reliability along with concomitant benefits for society and economy. It is anticipated these changes will soon affect household activity patterns, public safety, supply chains and logistics, manufacturing, and quality of life in general.

cs.CY↗

arXiv2015

The Stan Math Library: Reverse-Mode Automatic Differentiation in C++

As computational challenges in optimization and statistical inference grow ever harder, algorithms that utilize derivatives are becoming increasingly more important. The implementation of the derivatives that make these algorithms so powerful, however, is a substantial user burden and the practicality of these algorithms depends critically on tools like automatic differentiation that remove the implementation burden entirely. The Stan Math Library is a C++, reverse-mode automatic differentiation library designed to be usable, extensive and extensible, efficient, scalable, stable, portable, and redistributable in order to facilitate the construction and utilization of such algorithms. Usability is achieved through a simple direct interface and a cleanly abstracted functional interface. The extensive built-in library includes functions for matrix operations, linear algebra, differential equation solving, and most common probability functions. Extensibility derives from a straightforward object-oriented framework for expressions, allowing users to easily create custom functions. Efficiency is achieved through a combination of custom memory management, subexpression caching, traits-based metaprogramming, and expression templates. Partial derivatives for compound functions are evaluated lazily for improved scalability. Stability is achieved by taking care with arithmetic precision in algebraic expressions and providing stable, compound functions where possible. For portability, the library is standards-compliant C++ (03) and has been tested for all major compilers for Windows, Mac OS X, and Linux.

cs.MS↗

arXiv2000

The diagonalization of quantum field Hamiltonians

We introduce a new diagonalization method called quasi-sparse eigenvector diagonalization which finds the most important basis vectors of the low energy eigenstates of a quantum Hamiltonian. It can operate using any basis, either orthogonal or non-orthogonal, and any sparse Hamiltonian, either Hermitian, non-Hermitian, finite-dimensional, or infinite-dimensional. The method is part of a new computational approach which combines both diagonalization and Monte Carlo techniques.

hep-th↗