arXiv subjects

John Miller

Publications and source records attributed to John Miller.

At least 37 records · Page 2Linked to original sources

arXiv2020

The Effect of Natural Distribution Shift on Question Answering Models

We build four new test sets for the Stanford Question Answering Dataset (SQuAD) and evaluate the ability of question-answering systems to generalize to new data. Our first test set is from the original Wikipedia domain and measures the extent to which existing systems overfit the original test set. Despite several years of heavy test set re-use, we find no evidence of adaptive overfitting. The remaining three test sets are constructed from New York Times articles, Reddit posts, and Amazon product reviews and measure robustness to natural distribution shifts. Across a broad range of models, we observe average performance drops of 3.8, 14.0, and 17.4 F1 points, respectively. In contrast, a strong human baseline matches or exceeds the performance of SQuAD models on the original domain and exhibits little to no drop in new domains. Taken together, our results confirm the surprising resilience of the holdout method and emphasize the need to move towards evaluation metrics that incorporate robustness to natural distribution shifts.

cs.LG↗

arXiv2020

Strategic Classification is Causal Modeling in Disguise

Consequential decision-making incentivizes individuals to strategically adapt their behavior to the specifics of the decision rule. While a long line of work has viewed strategic adaptation as gaming and attempted to mitigate its effects, recent work has instead sought to design classifiers that incentivize individuals to improve a desired quality. Key to both accounts is a cost function that dictates which adaptations are rational to undertake. In this work, we develop a causal framework for strategic adaptation. Our causal perspective clearly distinguishes between gaming and improvement and reveals an important obstacle to incentive design. We prove any procedure for designing classifiers that incentivize improvement must inevitably solve a non-trivial causal inference problem. Moreover, we show a similar result holds for designing cost functions that satisfy the requirements of previous work. With the benefit of hindsight, our results show much of the prior work on strategic classification is causal modeling in disguise.

cs.LG↗

arXiv2019

Explicit Pieri Inclusions

By the Pieri rule, the tensor product of an exterior power and a finite-dimensional irreducible representation of a general linear group has a multiplicity-free decomposition. The embeddings of the constituents are called Pieri inclusions and were first studied by Weyman in his thesis and described explicitly by Olver. More recently, these maps have appeared in the work of Eisenbud, Fløstad, and Weyman and of Sam and Weyman to compute pure free resolutions for classical groups. In this paper, we give a new closed form, non-recursive description of Pieri inclusions. For partitions with a bounded number of distinct parts, the resulting algorithm has polynomial time complexity whereas the previously known algorithm has exponential time complexity.

math.RT↗

arXiv2019

A compact actively damped vibration isolation platform for optical experiments in ultra-high vacuum

We present a tabletop six-axis vibration isolation system, compatible with Ultra-High Vacuum (UHV), which is actively damped and provides 25 dB of isolation at 10 Hz and 65 dB at 100 Hz. While this isolation platform has been primarily designed to support optics in the Laser Interferometer Gravitational-Wave Observatory (LIGO) detectors, it is suitable for a variety of applications. The system has been engineered to facilitate the construction and assembly process, while minimizing cost. The platform provides passive isolation for six degrees of freedom using a combination of vertical springs and horizontal pendula. It is instrumented with voice-coil actuators and optical shadow sensors to damp the resonances. All materials are compatible with stringent vacuum requirements. Thanks to its architecture, the system's footprint can be adapted to meet spatial requirements, while maximizing the dimensions of the optical table. Three units are currently operating for LIGO. We present the design of the system, controls principle, and experimental results.

physics.ins-det↗

arXiv2019

Model Similarity Mitigates Test Set Overuse

Excessive reuse of test data has become commonplace in today's machine learning workflows. Popular benchmarks, competitions, industrial scale tuning, among other applications, all involve test data reuse beyond guidance by statistical confidence bounds. Nonetheless, recent replication studies give evidence that popular benchmarks continue to support progress despite years of extensive reuse. We proffer a new explanation for the apparent longevity of test data: Many proposed models are similar in their predictions and we prove that this similarity mitigates overfitting. Specifically, we show empirically that models proposed for the ImageNet ILSVRC benchmark agree in their predictions well beyond what we can conclude from their accuracy levels alone. Likewise, models created by large scale hyperparameter search enjoy high levels of similarity. Motivated by these empirical observations, we give a non-asymptotic generalization bound that takes similarity into account, leading to meaningful confidence bounds in practical settings.

cs.LG↗

arXiv2019

Stable Recurrent Models

Stability is a fundamental property of dynamical systems, yet to this date it has had little bearing on the practice of recurrent neural networks. In this work, we conduct a thorough investigation of stable recurrent models. Theoretically, we prove stable recurrent neural networks are well approximated by feed-forward networks for the purpose of both inference and training by gradient descent. Empirically, we demonstrate stable recurrent models often perform as well as their unstable counterparts on benchmark sequence tasks. Taken together, these findings shed light on the effective power of recurrent networks and suggest much of sequence learning happens, or can be made to happen, in the stable regime. Moreover, our results help to explain why in many cases practitioners succeed in replacing recurrent models by feed-forward models.

cs.LG↗

arXiv2018

The Social Cost of Strategic Classification

Consequential decision-making typically incentivizes individuals to behave strategically, tailoring their behavior to the specifics of the decision rule. A long line of work has therefore sought to counteract strategic behavior by designing more conservative decision boundaries in an effort to increase robustness to the effects of strategic covariate shift. We show that these efforts benefit the institutional decision maker at the expense of the individuals being classified. Introducing a notion of social burden, we prove that any increase in institutional utility necessarily leads to a corresponding increase in social burden. Moreover, we show that the negative externalities of strategic classification can disproportionately harm disadvantaged groups in the population. Our results highlight that strategy-robustness must be weighed against considerations of social welfare and fairness.

cs.LG↗

arXiv2018

Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning

We present Deep Voice 3, a fully-convolutional attention-based neural text-to-speech (TTS) system. Deep Voice 3 matches state-of-the-art neural speech synthesis systems in naturalness while training ten times faster. We scale Deep Voice 3 to data set sizes unprecedented for TTS, training on more than eight hundred hours of audio from over two thousand speakers. In addition, we identify common error modes of attention-based speech synthesis networks, demonstrate how to mitigate them, and compare several different waveform synthesis methods. We also describe how to scale inference to ten million queries per day on one single-GPU server.

cs.SD↗

arXiv2017

Deep Voice 2: Multi-Speaker Neural Text-to-Speech

We introduce a technique for augmenting neural text-to-speech (TTS) with lowdimensional trainable speaker embeddings to generate different voices from a single model. As a starting point, we show improvements over the two state-ofthe-art approaches for single-speaker neural TTS: Deep Voice 1 and Tacotron. We introduce Deep Voice 2, which is based on a similar pipeline with Deep Voice 1, but constructed with higher performance building blocks and demonstrates a significant audio quality improvement over Deep Voice 1. We improve Tacotron by introducing a post-processing neural vocoder, and demonstrate a significant audio quality improvement. We then demonstrate our technique for multi-speaker speech synthesis for both Deep Voice 2 and Tacotron on two multi-speaker TTS datasets. We show that a single neural TTS system can learn hundreds of unique voices from less than half an hour of data per speaker, while achieving high audio quality synthesis and preserving the speaker identities almost perfectly.

cs.CL↗

arXiv2017

Globally Normalized Reader

Rapid progress has been made towards question answering (QA) systems that can extract answers from text. Existing neural approaches make use of expensive bi-directional attention mechanisms or score all possible answer spans, limiting scalability. We propose instead to cast extractive QA as an iterative search problem: select the answer's sentence, start word, and end word. This representation reduces the space of each search step and allows computation to be conditionally allocated to promising search paths. We show that globally normalizing the decision process and back-propagating through beam search makes this representation viable and learning efficient. We empirically demonstrate the benefits of this approach using our model, Globally Normalized Reader (GNR), which achieves the second highest single model performance on the Stanford Question Answering Dataset (68.4 EM, 76.21 F1 dev) and is 24.7x faster than bi-attention-flow. We also introduce a data-augmentation method to produce semantically valid examples by aligning named entities to a knowledge base and swapping them with new entities of the same type. This method improves the performance of all models considered in this work and is of independent interest for a variety of NLP tasks.

cs.CL↗

arXiv2017

Damping parametric instabilities in future gravitational wave detectors by means of electrostatic actuators

It has been suggested that the next generation of interferometric gravitational wave detectors may observe spontaneously excited parametric oscillatory instabilities. We present a method of actively suppressing any such instability through application of electrostatic forces to the interferometers' test masses. Using numerical methods we quantify the actuation force required to damp candidate instabilities and find that such forces are readily achievable. Our predictions are subsequently verified experimentally using prototype Advanced LIGO hardware, conclusively demonstrating the effectiveness of our approach.

physics.ins-det↗

arXiv2017

Decoherence and degradation of squeezed states in quantum filter cavities

Squeezed states of light have been successfully employed in interferometric gravitational-wave detectors to reduce quantum noise, thus becoming one of the most promising options for extending the astrophysical reach of the generation of detectors currently under construction worldwide. In these advanced instruments, quantum noise will limit sensitivity over the entire detection band. Therefore, to obtain the greatest benefit from squeezing, the injected squeezed state must be filtered using a long-storage-time optical resonator, or "filter cavity", so as to realise a frequency dependent rotation of the squeezed quadrature. Whilst the ultimate performance of a filter cavity is determined by its storage time, several practical decoherence and degradation mechanisms limit the experimentally achievable quantum noise reduction. In this paper we develop an analytical model to explore these mechanisms in detail. As an example, we apply our results to the 16 m filter cavity design currently under consideration for the Advanced LIGO interferometers.

physics.optics↗

arXiv2017

Control and tuning of a suspended Fabry-Perot cavity using digitally-enhanced heterodyne interferometry

We present the first demonstration of real-time closed-loop control and deterministic tuning of an independently suspended Fabry-Perot optical cavity using digitally-enhanced heterodyne interferometry, realising a peak sensitivity of $\sim$10 pm$/\sqrt{\mathrm{Hz}}$ over the 10-1000 Hz frequency band. The methods presented are readily extensible to multiple coupled cavities. As such, we anticipate that refinements of this technique may find application in future interferometric gravitational-wave detectors.

physics.optics↗

arXiv2017

Deep Voice: Real-time Neural Text-to-Speech

We present Deep Voice, a production-quality text-to-speech system constructed entirely from deep neural networks. Deep Voice lays the groundwork for truly end-to-end neural speech synthesis. The system comprises five major building blocks: a segmentation model for locating phoneme boundaries, a grapheme-to-phoneme conversion model, a phoneme duration prediction model, a fundamental frequency prediction model, and an audio synthesis model. For the segmentation model, we propose a novel way of performing phoneme boundary detection with deep neural networks using connectionist temporal classification (CTC) loss. For the audio synthesis model, we implement a variant of WaveNet that requires fewer parameters and trains faster than the original. By using a neural network for each component, our system is simpler and more flexible than traditional text-to-speech systems, where each component requires laborious feature engineering and extensive domain expertise. Finally, we show that inference with our system can be performed faster than real time and describe optimized WaveNet inference kernels on both CPU and GPU that achieve up to 400x speedups over existing implementations.

cs.CL↗

arXiv2016

First Demonstration of Electrostatic Damping of Parametric Instability at Advanced LIGO

Interferometric gravitational wave detectors operate with high optical power in their arms in order to achieve high shot-noise limited strain sensitivity. A significant limitation to increasing the optical power is the phenomenon of three-mode parametric instabilities, in which the laser field in the arm cavities is scattered into higher order optical modes by acoustic modes of the cavity mirrors. The optical modes can further drive the acoustic modes via radiation pressure, potentially producing an exponential buildup. One proposed technique to stabilize parametric instability is active damping of acoustic modes. We report here the first demonstration of damping a parametrically unstable mode using active feedback forces on the cavity mirror. A 15,538 Hz mode that grew exponentially with a time constant of 182 sec was damped using electro-static actuation, with a resulting decay time constant of 23 sec. An average control force of 0.03 nNrms was required to maintain the acoustic mode at its minimum amplitude.

physics.ins-det↗

arXiv2016

Observing the carbon-climate system

Increases in atmospheric CO2 and CH4 result from a combination of forcing from anthropogenic emissions and Earth System feedbacks that reduce or amplify the effects of those emissions on atmospheric concentrations. Despite decades of research carbon-climate feedbacks remain poorly quantified. The impact of these uncertainties on future climate are of increasing concern, especially in the wake of recent climate negotiations. Emissions, long concentrated in the developed world, are now shifting to developing countries, where the emissions inventories have larger uncertainties. The fraction of anthropogenic CO2 remaining in the atmosphere has remained remarkably constant over the last 50 years. Will this change in the future as the climate evolves? Concentrations of CH4, the 2nd most important greenhouse gas, which had apparently stabilized, have recently resumed their increase, but the exact cause for this is unknown. While greenhouse gases affect the global atmosphere, their sources and sinks are remarkably heterogeneous in time and space, and traditional in situ observing systems do not provide the coverage and resolution to attribute the changes to these greenhouse gases to specific sources or sinks. In the past few years, space-based technologies have shown promise for monitoring carbon stocks and fluxes. Advanced versions of these capabilities could transform our understanding and provide the data needed to quantify carbon-climate feedbacks. A new observing system that allows resolving global high resolution fluxes will capture variations on time and space scales that allow the attribution of these fluxes to underlying mechanisms.

physics.ao-ph↗

arXiv2015

Traversing Knowledge Graphs in Vector Space

Path queries on a knowledge graph can be used to answer compositional questions such as "What languages are spoken by people living in Lisbon?". However, knowledge graphs often have missing facts (edges) which disrupts path queries. Recent models for knowledge base completion impute missing facts by embedding knowledge graphs in vector spaces. We show that these models can be recursively applied to answer path queries, but that they suffer from cascading errors. This motivates a new "compositional" training objective, which dramatically improves all models' ability to answer path queries, in some cases more than doubling accuracy. On a standard knowledge base completion task, we also demonstrate that compositional training acts as a novel form of structural regularization, reliably improving performance across all base models (reducing errors by up to 43%) and achieving new state-of-the-art results.

cs.CL↗

arXiv2015

Audio-band frequency-dependent squeezing

Quantum vacuum fluctuations impose strict limits on precision displacement measurements, those of interferometric gravitational-wave detectors among them. Introducing squeezed states into an interferometer's readout port can improve the sensitivity of the instrument, leading to richer astrophysical observations. However, optomechanical interactions dictate that the vacuum's squeezed quadrature must rotate by 90 degrees around 50Hz. Here we use a 2-m-long, high-finesse optical resonator to produce frequency-dependent rotation around 1.2kHz. This demonstration of audio-band frequency-dependent squeezing uses technology and methods that are scalable to the required rotation frequency, heralding application of the technique in future gravitational-wave detectors.

quant-ph↗