arXiv subjects

Andrew Lee

Publications and source records attributed to Andrew Lee.

54 records · Page 3Linked to original sources

arXiv2022

Improving Chess Commentaries by Combining Language Models with Symbolic Reasoning Engines

Despite many recent advancements in language modeling, state-of-the-art language models lack grounding in the real world and struggle with tasks involving complex reasoning. Meanwhile, advances in the symbolic reasoning capabilities of AI have led to systems that outperform humans in games like chess and Go (Silver et al., 2018). Chess commentary provides an interesting domain for bridging these two fields of research, as it requires reasoning over a complex board state and providing analyses in natural language. In this work we demonstrate how to combine symbolic reasoning engines with controllable language models to generate chess commentaries. We conduct experiments to demonstrate that our approach generates commentaries that are preferred by human judges over previous baselines.

cs.CL↗

arXiv2022

Augmenting Task-Oriented Dialogue Systems with Relation Extraction

The standard task-oriented dialogue pipeline uses intent classification and slot-filling to interpret user utterances. While this approach can handle a wide range of queries, it does not extract the information needed to handle more complex queries that contain relationships between slots. We propose integration of relation extraction into this pipeline as an effective way to expand the capabilities of dialogue systems. We evaluate our approach by using an internal dataset with slot and relation annotations spanning three domains. Finally, we show how slot-filling annotation schemes can be simplified once the expressive power of relation annotations is available, reducing the number of slots while still capturing the user's intended meaning.

cs.CL↗

arXiv2022

Sharp bounds on the price of bandit feedback for several models of mistake-bounded online learning

We determine sharp bounds on the price of bandit feedback for several variants of the mistake-bound model. The first part of the paper presents bounds on the $r$-input weak reinforcement model and the $r$-input delayed, ambiguous reinforcement model. In both models, the adversary gives $r$ inputs in each round and only indicates a correct answer if all $r$ guesses are correct. The only difference between the two models is that in the delayed, ambiguous model, the learner must answer each input before receiving the next input of the round, while the learner receives all $r$ inputs at once in the weak reinforcement model. In the second part of the paper, we introduce models for online learning with permutation patterns, in which a learner attempts to learn a permutation from a set of permutations by guessing statistics related to sub-permutations. For these permutation models, we prove sharp bounds on the price of bandit feedback.

cs.LG↗

arXiv2022

Diversifying the Genomic Data Science Research Community

Over the last 20 years, there has been an explosion of genomic data collected for disease association, functional analyses, and other large-scale discoveries. At the same time, there have been revolutions in cloud computing that enable computational and data science research, while making data accessible to anyone with a web browser and an internet connection. However, students at institutions with limited resources have received relatively little exposure to curricula or professional development opportunities that lead to careers in genomic data science. To broaden participation in genomics research, the scientific community needs to support students, faculty, and administrators at Underserved Institutions (UIs) including Community Colleges, Historically Black Colleges and Universities, Hispanic-Serving Institutions, and Tribal Colleges and Universities in taking advantage of these tools in local educational and research programs. We have formed the Genomic Data Science Community Network (http://www.gdscn.org/) to identify opportunities and support broadening access to cloud-enabled genomic data science. Here, we provide a summary of the priorities for faculty members at UIs, as well as administrators, funders, and R1 researchers to consider as we create a more diverse genomic data science community.

q-bio.OT↗

arXiv2022

Hypergraph Fuss-Catalan Numbers

The Catalan numbers $C_n$ are an extremely well-studied sequence of numbers that appear as the answer to many combinatorial problems. Two generalizations of these numbers that have been studied are the Fuss-Catalan numbers and the Hypergraph Catalan numbers. In this paper, we study the combination of these, the Hypergraph Fuss-Catalan numbers. We provide some combinatorial interpretations of these numbers, as well as describe their generating function.

math.CO↗

arXiv2021

Micromodels for Efficient, Explainable, and Reusable Systems: A Case Study on Mental Health

Many statistical models have high accuracy on test benchmarks, but are not explainable, struggle in low-resource scenarios, cannot be reused for multiple tasks, and cannot easily integrate domain expertise. These factors limit their use, particularly in settings such as mental health, where it is difficult to annotate datasets and model outputs have significant impact. We introduce a micromodel architecture to address these challenges. Our approach allows researchers to build interpretable representations that embed domain knowledge and provide explanations throughout the model's decision process. We demonstrate the idea on multiple mental health tasks: depression classification, PTSD classification, and suicidal risk assessment. Our systems consistently produce strong results, even in low-resource scenarios, and are more interpretable than alternative methods.

cs.CL↗

arXiv2020

Balance Regularized Neural Network Models for Causal Effect Estimation

Estimating individual and average treatment effects from observational data is an important problem in many domains such as healthcare and e-commerce. In this paper, we advocate balance regularization of multi-head neural network architectures. Our work is motivated by representation learning techniques to reduce differences between treated and untreated distributions that potentially arise due to confounding factors. We further regularize the model by encouraging it to predict control outcomes for individuals in the treatment group that are similar to control outcomes in the control group. We empirically study the bias-variance trade-off between different weightings of the regularizers, as well as between inductive and transductive inference.

cs.LG↗

arXiv2020

Mechanical Search on Shelves using Lateral Access X-RAY

Efficiently finding an occluded object with lateral access arises in many contexts such as warehouses, retail, healthcare, shipping, and homes. We introduce LAX-RAY (Lateral Access maXimal Reduction of occupancY support Area), a system to automate the mechanical search for occluded objects on shelves. For such lateral access environments, LAX-RAY couples a perception pipeline predicting a target object occupancy support distribution with a mechanical search policy that sequentially selects occluding objects to push to the side to reveal the target as efficiently as possible. Within the context of extruded polygonal objects and a stationary target with a known aspect ratio, we explore three lateral access search policies: Distribution Area Reduction (DAR), Distribution Entropy Reduction (DER), and Distribution Entropy Reduction over Multiple Time Steps (DER-MT) utilizing the support distribution and prior information. We evaluate these policies using the First-Order Shelf Simulator (FOSS) in which we simulate 800 random shelf environments of varying difficulty, and in a physical shelf environment with a Fetch robot and an embedded PrimeSense RGBD Camera. Average simulation results of 87.3% success rate demonstrate better performance of DER-MT with 2 prediction steps. When deployed on the robot, results show a success rate of at least 80% for all policies, suggesting that LAX-RAY can efficiently reveal the target object in reality. Both results show significantly better performance of the three proposed policies compared to a baseline policy with uniform probability distribution assumption in non-trivial cases, showing the importance of distribution prediction. Code, videos, and supplementary material can be found at https://sites.google.com/berkeley.edu/lax-ray.

cs.RO↗

arXiv2020

Analyzing and Improving Neural Networks by Generating Semantic Counterexamples through Differentiable Rendering

Even as deep neural networks (DNNs) have achieved remarkable success on vision-related tasks, their performance is brittle to transformations in the input. Of particular interest are semantic transformations that model changes that have a basis in the physical world, such as rotations, translations, changes in lighting or camera pose. In this paper, we show how differentiable rendering can be utilized to generate images that are informative, yet realistic, and which can be used to analyze DNN performance and improve its robustness through data augmentation. Given a differentiable renderer and a DNN, we show how to use off-the-shelf attacks from adversarial machine learning to generate semantic counterexamples -- images where semantic features are changed as to produce misclassifications or misdetections. We validate our approach on DNNs for image classification and object detection. For classification, we show that semantic counterexamples, when used to augment the dataset, (i) improve generalization performance (ii) enhance robustness to semantic transformations, and (iii) transfer between models. Additionally, in comparison to sampling-based semantic augmentation, our technique generates more informative data in a sample efficient manner.

cs.LG↗

arXiv2020

Exactly computing the tail of the Poisson-Binomial Distribution

We offer ShiftConvolvePoibin, a fast exact method to compute the tail of a Poisson-Binomial distribution (PBD). Our method employs an exponential shift to retain its accuracy when computing a tail probability, and in practice we find that it is immune to the significant relative errors that other methods, exact or approximate, can suffer from when computing very small tail probabilities of the PBD. The accompanying R package is also competitive with the fastest implementations for computing the entire PBD.

stat.ME↗

arXiv2019

An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction

Task-oriented dialog systems need to know when a query falls outside their range of supported intents, but current text classification corpora only define label sets that cover every example. We introduce a new dataset that includes queries that are out-of-scope---i.e., queries that do not fall into any of the system's supported intents. This poses a new challenge because models cannot assume that every query at inference time belongs to a system-supported intent class. Our dataset also covers 150 intent classes over 10 domains, capturing the breadth that a production task-oriented agent must handle. We evaluate a range of benchmark classifiers on our dataset along with several different out-of-scope identification schemes. We find that while the classifiers perform well on in-scope intent classification, they struggle to identify out-of-scope queries. Our dataset and evaluation fill an important gap in the field, offering a way of more rigorously and realistically benchmarking text classification in task-driven dialog systems.

cs.CL↗

arXiv2019

Outlier Detection for Improved Data Quality and Diversity in Dialog Systems

In a corpus of data, outliers are either errors: mistakes in the data that are counterproductive, or are unique: informative samples that improve model robustness. Identifying outliers can lead to better datasets by (1) removing noise in datasets and (2) guiding collection of additional data to fill gaps. However, the problem of detecting both outlier types has received relatively little attention in NLP, particularly for dialog systems. We introduce a simple and effective technique for detecting both erroneous and unique samples in a corpus of short texts using neural sentence embeddings combined with distance-based outlier detection. We also present a novel data collection pipeline built atop our detection technique to automatically and iteratively mine unique data samples while discarding erroneous samples. Experiments show that our outlier detection technique is effective at finding errors while our data collection pipeline yields highly diverse corpora that in turn produce more robust intent classification and slot-filling models.

cs.CL↗

arXiv2019

Segmenting Unknown 3D Objects from Real Depth Images using Mask R-CNN Trained on Synthetic Data

The ability to segment unknown objects in depth images has potential to enhance robot skills in grasping and object tracking. Recent computer vision research has demonstrated that Mask R-CNN can be trained to segment specific categories of objects in RGB images when massive hand-labeled datasets are available. As generating these datasets is time consuming, we instead train with synthetic depth images. Many robots now use depth sensors, and recent results suggest training on synthetic depth data can transfer successfully to the real world. We present a method for automated dataset generation and rapidly generate a synthetic training dataset of 50,000 depth images and 320,000 object masks using simulated heaps of 3D CAD models. We train a variant of Mask R-CNN with domain randomization on the generated dataset to perform category-agnostic instance segmentation without any hand-labeled data and we evaluate the trained network, which we refer to as Synthetic Depth (SD) Mask R-CNN, on a set of real, high-resolution depth images of challenging, densely-cluttered bins containing objects with highly-varied geometry. SD Mask R-CNN outperforms point cloud clustering baselines by an absolute 15% in Average Precision and 20% in Average Recall on COCO benchmarks, and achieves performance levels similar to a Mask R-CNN trained on a massive, hand-labeled RGB dataset and fine-tuned on real images from the experimental setup. We deploy the model in an instance-specific grasping pipeline to demonstrate its usefulness in a robotics application. Code, the synthetic training dataset, and supplementary material are available at https://bit.ly/2letCuE.

cs.CV↗

arXiv2010

The b quark mass from lattice nonrelativistic QCD

We present the first two-loop calculation of the heavy quark energy shift in lattice nonrelativistic QCD (NRQCD). This calculation allow us to extract a preliminary prediction of $m_b(m_b, n_f = 5) = 4.25(12)$ GeV for the mass of the b quark from lattice NRQCD simulations performed with a lattice of spacing $a=0.12$fm. Our result is an improvement on a previous determination of the b quark mass from unquenched lattice NRQCD simulations, which was limited by the use of one-loop expressions for the energy shift. Our value is in good agreement with recent results of $m_b(m_b) = 4.163(16)$ GeV from QCD sum rules and $m_b(m_b, n_f = 5) = 4.170(25)$ GeV from realistic lattice simulations using highly-improved staggered quarks. We employ a mixed strategy to simplify our calculation. Ghost, gluon and counterterm contributions to the energy shift and mass renormalisation are extracted from quenched high-beta simulations whilst fermionic contributions are calculated using automated lattice perturbation theory. Our results demonstrate the effectiveness of such a strategy.

hep-lat↗

arXiv2009

Radiative corrections to the m(oving)NRQCD action and heavy-light operators

Rare decays of B mesons, such as B \to K^*γand B\to K^{(*)}\ell^+\ell^- are loop suppressed in the Standard Model and sensitive to new physics. The final state meson in heavy-light decays at large recoil has sizeable momentum in the rest frame of the decaying meson. To reduce the resulting discretization errors we formulate the nonrelativistic heavy quark action in a moving frame. We discuss the perturbative renormalization of the leading order heavy-light operators in the resulting theory which is known as m(oving)NRQCD. We also present radiative corrections to the NRQCD action computed using automated lattice perturbation theory. By combining this technique with high-beta simulations in the weak coupling regime of the theory higher order loop corrections can be calculated very efficiently.

hep-lat↗

arXiv2000

Properties of Gamma-Ray Burst Time Profiles Using Pulse Decomposition Analysis

The time profiles of many gamma-ray bursts consist of distinct pulses, which offers the possibility of characterizing the temporal structure of these bursts using a relatively small set of pulse shape parameters. This pulse decomposition analysis has previously been performed on a small sample of bright long bursts using binned data from BATSE, which comes in several data types, and on a sample of short bursts using the BATSE Time-Tagged Event (TTE) data type. We have developed an interactive pulse-fitting program using the phenomenological pulse model of Norris, et al. and a maximum-likelihood fitting routine. We have used this program to analyze the Time-to-Spill (TTS) data for all bursts observed by BATSE up through trigger number 2000, in all energy channels for which TTS data is available. We present statistical information on the attributes of pulses comprising these bursts, including relations between pulse characteristics in different energy channels and the evolution of pulse characteristics through the course of a burst. We carry out simulations to determine the biases that our procedures may introduce. We find that pulses tend to have shorter rise times than decay times, and tend to be narrower and peak earlier at higher energies. We also find that pulse brightness, pulse width, and pulse hardness ratios do not evolve monotonically within bursts, but that the ratios of pulse rise times to decay times tend to decrease with time within bursts.

astro-ph↗

arXiv2000

Intrinsic and Cosmological Signatures in Gamma-Ray Burst Time Profiles: Time Dilation

The time profiles of many gamma-ray bursts consist of distinct pulses, which offers the possibility of characterizing the temporal structure of these bursts using a relatively small set of pulse shape parameters. We have used a pulse decomposition procedure to analyze the Time-to-Spill (TTS) data for all bursts observed by BATSE up through trigger number 2000, in all energy channels for which TTS data is available. We obtain amplitude, rise and decay timescales, a pulse shape parameter, and the fluences of individual pulses in all of the bursts. We investigate the correlations between brightness measures (amplitude and fluence) and timescale measures (pulse width and separation) which may result from cosmological time dilation of bursts, or from intrinsic properties of burst sources or from selection effects. The effects of selection biases are evaluated through simulations. The correlations between these parameters among pulses within individual bursts give a measure of the intrinsic effects while the correlations among bursts could result both from intrinsic and cosmological effects. We find that timescales tend to be shorter in bursts with higher peak fluxes, as expected from cosmological time dilation effects, but also find that there are non-cosmological effects contributing to this inverse correlation. We find that timescales tend to be longer in bursts with higher total fluences, contrary to what is expected from cosmological effects. We also find that peak fluxes and total fluences of bursts are uncorrelated, indicating that they cannot both be good distance indicators for bursts.

astro-ph↗

arXiv1999

Cosmological Signatures in Temporal and Spectral Characteristics of Gamma-Ray Bursts

There have been several suggestions of the existence of cosmological redshift signatures in the temporal and spectral characteristics of gamma-ray bursts. However, recent discoveries of afterglows and redshift measurements indicate the presence of broad ``luminosity functions'' which may overwhelm such weaker cosmological signatures. The primary goal of this paper is to determine if the intrinsic and cosmological dispersions can be separated. We have expanded the search for cosmological signatures to several other temporal and spectral features, have determined correlations which could arise from the cosmological redshift of the sources and have carried out tests to determine if the observed correlations can be due to cosmology. We find that the intrinsic dispersions are the dominant factors and that detection of cosmological signatures must await accumulation of a much larger number of identification with galaxies and measurements of redshifts.

astro-ph↗