arXiv subjects

Xiaodong Liu

Publications and source records attributed to Xiaodong Liu.

At least 199 records · Page 11Linked to original sources

arXiv2019

Multi-task Learning with Sample Re-weighting for Machine Reading Comprehension

We propose a multi-task learning framework to learn a joint Machine Reading Comprehension (MRC) model that can be applied to a wide range of MRC tasks in different domains. Inspired by recent ideas of data selection in machine translation, we develop a novel sample re-weighting scheme to assign sample-specific weights to the loss. Empirical study shows that our approach can be applied to many existing MRC models. Combined with contextual representations from pre-trained language models (such as ELMo), we achieve new state-of-the-art results on a set of MRC benchmark datasets. We release our code at https://github.com/xycforgithub/MultiTask-MRC.

cs.CL↗

arXiv2019

Stochastic Answer Networks for Natural Language Inference

We propose a stochastic answer network (SAN) to explore multi-step inference strategies in Natural Language Inference. Rather than directly predicting the results given the inputs, the model maintains a state and iteratively refines its predictions. Our experiments show that SAN achieves the state-of-the-art results on three benchmarks: Stanford Natural Language Inference (SNLI) dataset, MultiGenre Natural Language Inference (MultiNLI) dataset and Quora Question Pairs dataset.

cs.CL↗

arXiv2018

Inverse elastic scattering problems with phaseless far field data

This paper is concerned with uniqueness, phase retrieval and shape reconstruction methods for inverse elastic scattering problems with phaseless far field data. Systematically, we study two basic models, i.e., inverse scattering of plane waves by rigid bodies and inverse scattering of sources with compact support. For both models, we show that the location of the objects can not be uniquely recovered by the data. To solve this problem, we consider simultaneously the incident point sources with one fixed source point and at most three scattering strengths. We then establish some uniqueness results for source scattering problem with multi-frequency phaseless far field data. Furthermore, a fast and stable phase retrieval approach is proposed based on a simple geometric result which provides a stable reconstruction of a point in the plane from three distances to given points. Difficulties arise for inverse scattering by rigid bodies due to the additional unknown far field pattern of the point sources. To overcome this difficulty, we introduce an artificial rigid body into the system and show that the underlying rigid bodies can be uniquely determined by the corresponding phaseless far field data at a fixed frequency. Noting that the far field pattern of the scattered field corresponding to point sources is very small if the source point is far away from the scatterers, we propose an appropriate phase retrieval method for obstacle scattering problems, without using the artificial rigid body. Finally, we propose several sampling methods for shape reconstruction with phaseless far field data. Extended numerical examples in two dimensions are conducted with noisy data, and the results further verify the effectiveness and robustness of the proposed phase retrieval techniques and sampling methods.

math.AP↗

arXiv2018

MS MARCO: A Human Generated MAchine Reading COmprehension Dataset

We introduce a large scale MAchine Reading COmprehension dataset, which we name MS MARCO. The dataset comprises of 1,010,916 anonymized questions---sampled from Bing's search query logs---each with a human generated answer and 182,669 completely human rewritten generated answers. In addition, the dataset contains 8,841,823 passages---extracted from 3,563,535 web documents retrieved by Bing---that provide the information necessary for curating the natural language answers. A question in the MS MARCO dataset may have multiple answers or no answers at all. Using this dataset, we propose three different tasks with varying levels of difficulty: (i) predict if a question is answerable given a set of context passages, and extract and synthesize the answer as a human would (ii) generate a well-formed answer (if possible) based on the context passages that can be understood with the question and passage context, and finally (iii) rank a set of retrieved passages given a question. The size of the dataset and the fact that the questions are derived from real user search queries distinguishes MS MARCO from other well-known publicly available datasets for machine reading comprehension and question-answering. We believe that the scale and the real-world nature of this dataset makes it attractive for benchmarking machine reading comprehension and question-answering models.

cs.CL↗

arXiv2018

ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension

We present a large-scale dataset, ReCoRD, for machine reading comprehension requiring commonsense reasoning. Experiments on this dataset demonstrate that the performance of state-of-the-art MRC systems fall far behind human performance. ReCoRD represents a challenge for future research to bridge the gap between human and machine commonsense reading comprehension. ReCoRD is available at http://nlp.jhu.edu/record.

cs.CL↗

arXiv2018

Stochastic Answer Networks for SQuAD 2.0

This paper presents an extension of the Stochastic Answer Network (SAN), one of the state-of-the-art machine reading comprehension models, to be able to judge whether a question is unanswerable or not. The extended SAN contains two components: a span detector and a binary classifier for judging whether the question is unanswerable, and both components are jointly optimized. Experiments show that SAN achieves the results competitive to the state-of-the-art on Stanford Question Answering Dataset (SQuAD) 2.0. To facilitate the research on this field, we release our code: https://github.com/kevinduh/san_mrc.

cs.CL↗

arXiv2018

Dust in the Jupiter system outside the rings

Jupiter is one of the major targets for planetary exploration, and dust in the Jovian system is of great interest to researchers in the field of planetary science. In this paper, we review the five dust populations outside the ring system: grains in the region of the Galilean moons, potential dust from plumes on Europa, Jovian stream particles, particles in the outer region of the Jovian system ejected from the irregular satellites, and dust in the region of the Trojan asteroids. The physical environment for the dust dynamics is described, including the gravity, the magnetic field and the plasma environment. For each population, the dust sources are described, and the relevant perturbation forces are discussed. Observations and results from modeling are reviewed, and the distributions of the individual dust populations are shown. The understanding of the Jovian dust environment allows to assess the dust hazard to spacecraft, and to characterize the material exchange between the Jovian moons, their surface properties and distribution of non-icy constituents.

astro-ph.EP↗

arXiv2018

Target reconstruction with a reference point scatterer using phaseless far field patterns

An important property of the phaseless far field patterns with incident plane waves is the translation invariance. Thus it is impossible to reconstruct the location of the underlying scatterers. By adding a reference point scatterer into the model, we design a novel direct sampling method using the phaseless data directly. The reference point technique not only overcomes the translation invariance, but also brings a practical phase retrieval algorithm. Based on this, we propose a hybrid method combining the novel phase retrieval algorithm and the classical direct sampling methods. Numerical examples in two dimensions are presented to demonstrate their effectiveness and robustness.

math.AP↗

arXiv2018

Phaseless inverse source scattering problem: phase retrieval, uniqueness and direct sampling methods

Similar to the obstacle or medium scattering problems, an important property of the phaseless far field patterns for source scattering problems is the translation invariance. Thus it is impossible to reconstruct the location of the underlying sources. Furthermore, the phaseless far field pattern is also invariant if the source is multiplied by any complex number with modulus one. Therefore, the source can not be uniquely determined, even the multifrequency phaseless far field patterns are considered. By adding a reference point source into the model, we propose a simple and stable phase retrieval method and establish several uniqueness results with phaseless far field data. We proceed to introduce a novel direct sampling method for shape and location reconstruction of the source by using broadband sparse phaseless data directly. We also propose a combination method with the novel phase retrieval algorithm and the classical direct sampling methods with phased data. Numerical examples in two dimensions are also presented to demonstrate their feasibility and effectiveness.

math.AP↗

arXiv2018

Navigating with Graph Representations for Fast and Scalable Decoding of Neural Language Models

Neural language models (NLMs) have recently gained a renewed interest by achieving state-of-the-art performance across many natural language processing (NLP) tasks. However, NLMs are very computationally demanding largely due to the computational cost of the softmax layer over a large vocabulary. We observe that, in decoding of many NLP tasks, only the probabilities of the top-K hypotheses need to be calculated preciously and K is often much smaller than the vocabulary size. This paper proposes a novel softmax layer approximation algorithm, called Fast Graph Decoder (FGD), which quickly identifies, for a given context, a set of K words that are most likely to occur according to a NLM. We demonstrate that FGD reduces the decoding time by an order of magnitude while attaining close to the full softmax baseline accuracy on neural machine translation and language modeling tasks. We also prove the theoretical guarantee on the softmax approximation quality.

cs.CL↗

arXiv2018

Language-Based Image Editing with Recurrent Attentive Models

We investigate the problem of Language-Based Image Editing (LBIE). Given a source image and a natural language description, we want to generate a target image by editing the source image based on the description. We propose a generic modeling framework for two sub-tasks of LBIE: language-based image segmentation and image colorization. The framework uses recurrent attentive models to fuse image and language features. Instead of using a fixed step size, we introduce for each region of the image a termination gate to dynamically determine after each inference step whether to continue extrapolating additional information from the textual description. The effectiveness of the framework is validated on three datasets. First, we introduce a synthetic dataset, called CoSaL, to evaluate the end-to-end performance of our LBIE system. Second, we show that the framework leads to state-of-the-art performance on image segmentation on the ReferIt dataset. Third, we present the first language-based colorization result on the Oxford-102 Flowers dataset.

cs.CV↗

arXiv2018

Stable Periodic Orbits for Spacecrafts around Minor Celestial Bodies

We are interested in stable periodic orbits for spacecrafts in the gravitational field of minor celestial bodies. The stable periodic orbits around minor celestial bodies are useful not only for the mission design of the deep space exploration, but also for studying the long-time stability of small satellites in the large-size-ratio binary asteroids. The irregular shapes and gravitational fields of the minor celestial bodies are modeled by the polyhedral model. Using the topological classifications of periodic orbits and the grid search method, the stable periodic orbits can be calculated and the topological cases can be determined. Furthermore, we find five different types of stable periodic orbits around minor celestial bodies: A) Stable periodic orbits generated from the stable equilibrium points outside the minor celestial body, B) Stable periodic orbits continued from the unstable periodic orbits around the unstable equilibrium points, C) Retrograde and nearly circular periodic orbits with zero-inclination around minor celestial bodies, D) Resonance periodic orbits, E) Near-surface inclined periodic orbits. We take asteroid 243 Ida, 433 Eros, 6489 Golevka, 101955 Bennu, and the comet 1P/Halley for examples.

astro-ph.EP↗

arXiv2018

On a novel inverse scattering scheme using resonant modes with enhanced imaging resolution

We develop a novel wave imaging scheme for reconstructing the shape of an inhomogeneous scatterer and we consider the inverse acoustic obstacle scattering problem as a prototype model for our study. There exists a wealth of reconstruction methods for the inverse obstacle scattering problem and many of them intentionally avoid the interior resonant modes. Indeed, the occurrence of the interior resonance may cause the failure of the corresponding reconstruction. However, based on the observation that the interior resonant modes actually carry the geometrical information of the underlying obstacle, we propose an inverse scattering scheme of using those resonant modes for the reconstruction. To that end, we first develop a numerical procedure in determining the interior eigenvalues associated with an unknown obstacle from its far-field data based on the validity of the factorization method. Then we propose two efficient optimization methods in further determining the corresponding eigenfunctions. Using the afore-determined interior resonant modes, we show that the shape of the underlying obstacle can be effectively recovered. Moreover, the reconstruction yields enhanced imaging resolution, especially for the concave part of the obstacle. We provide rigorous theoretical justifications for the proposed method. Numerical examples in 2D and 3D verify the theoretically predicted effectiveness and efficiency of the method.

math.AP↗

arXiv2018

Stochastic Answer Networks for Machine Reading Comprehension

We propose a simple yet robust stochastic answer network (SAN) that simulates multi-step reasoning in machine reading comprehension. Compared to previous work such as ReasoNet which used reinforcement learning to determine the number of steps, the unique feature is the use of a kind of stochastic prediction dropout on the answer module (final layer) of the neural network during the training. We show that this simple trick improves robustness and achieves results competitive to the state-of-the-art on the Stanford Question Answering Dataset (SQuAD), the Adversarial SQuAD, and the Microsoft MAchine Reading COmprehension Dataset (MS MARCO).

cs.CL↗

arXiv2018

Comparison of the orbital properties of Jupiter Trojan asteroids and Trojan dust

In a previous paper we simulated the orbital evolution of dust particles from the Jupiter Trojan asteroids ejected by the impacts of interplanetary particles, and evaluated their overall configuration in the form of dust arcs. Here we compare the orbital properties of these Trojan dust particles and the Trojan asteroids. Both Trojan asteroids and most of the dust particles are trapped in the Jupiter 1:1 resonance. However, for dust particles, this resonance is modified because of the presence of solar radiation pressure, which reduces the peak value of the semi-major axis distribution. We find also that some particles can be trapped in the Saturn 1:1 resonance and higher order resonances with Jupiter. The distributions of the eccentricity, the longitude of pericenter, and the inclination for Trojans and the dust are compared. For the Trojan asteroids, the peak in the longitude of pericenter distribution is about 60 degrees larger than the longitude of pericenter of Jupiter; in contrast, for Trojan dust this difference is smaller than 60 degrees, and it decreases with decreasing grain size. For the Trojan asteroids and most of the Trojan dust, the Tisserand parameter is distributed in the range of two to three.

astro-ph.EP↗

arXiv2018

Dynamic Fusion Networks for Machine Reading Comprehension

This paper presents a novel neural model - Dynamic Fusion Network (DFN), for machine reading comprehension (MRC). DFNs differ from most state-of-the-art models in their use of a dynamic multi-strategy attention process, in which passages, questions and answer candidates are jointly fused into attention vectors, along with a dynamic multi-step reasoning module for generating answers. With the use of reinforcement learning, for each input sample that consists of a question, a passage and a list of candidate answers, an instance of DFN with a sample-specific network architecture can be dynamically constructed by determining what attention strategy to apply and how many reasoning steps to take. Experiments show that DFNs achieve the best result reported on RACE, a challenging MRC dataset that contains real human reading questions in a wide variety of types. A detailed empirical analysis also demonstrates that DFNs can produce attention vectors that summarize information from questions, passages and answer candidates more effectively than other popular MRC models.

cs.CL↗

arXiv2018

Dynamics and distribution of Jovian dust ejected from the Galilean satellites

In this paper, the dynamical analysis of the Jovian dust originating from the four Galilean moons is presented. High accuracy orbital integrations of dust particles are used to determine their dynamical evolution. A variety of forces are taken into account, including the Lorentz force, solar radiation pressure, Poynting-Robertson drag, solar gravity, the satellites' gravity, plasma drag, and gravitational effects due to non-sphericity of Jupiter. More than 20,000 dust particles from each source moon in the size range from 0.05 micron to 1 cm are simulated over 8,000 (Earth) years until each dust grain hits a sink (moons, Jupiter, or escape from the system). Configurations of dust number density in the Jovicentric equatorial inertial frame are calculated and shown. In a Jovicentric frame rotating with the Sun the dust distributions are found to be asymmetric. For certain small particle sizes, the dust population is displaced towards the Sun, while for certain larger sizes, the dust population is displaced away from the Sun. The average lifetime as a function of particle size for ejecta from each source moon is derived, and two sharp jumps in the average lifetime are analyzed. Transport of dust between the Galilean moons and to Jupiter is investigated. Most of the orbits for dust particles from Galilean moons are prograde, while, surprisingly, a small fraction of orbits are found to become retrograde mainly due to solar radiation pressure and Lorentz force. The distribution of orbital elements is also analyzed.

astro-ph.EP↗

arXiv2018

Dust arcs in the region of Jupiter's Trojan asteroids

Aims. The surfaces of the Trojan asteroids are steadily bombarded by interplanetary micrometeoroids, which releases ejecta of small dust particles. These particles form the faint dust arcs that are associated with asteroid clouds. Here we analyze the particle dynamics and structure of the arc in the region of the $L_4$ Trojan asteroids. Methods. We calculate the total cross section of the $L_4$ Trojan asteroids and the production rate of dust particles. The motion of the particles is perturbed by a variety of forces. We simulate the dynamical evolution of the dust particles, and explore the overall features of the Trojan dust arc. Results. The simulations show that the arc is mainly composed of grains in the size range 4-10 microns. Compared to the $L_4$ Trojan asteroids, the dust arc is distributed more widely in the azimuthal direction, extending to a range of [30,120] degrees relative to Jupiter. The peak number density does not develop at $L_4$. There exist two peaks that are azimuthally displaced from $L_4$.

astro-ph.EP↗