arXiv subjects

Peng Qian

Publications and source records attributed to Peng Qian.

47 records · Page 3Linked to original sources

arXiv2020

A Systematic Assessment of Syntactic Generalization in Neural Language Models

While state-of-the-art neural network models continue to achieve lower perplexity scores on language modeling benchmarks, it remains unknown whether optimizing for broad-coverage predictive performance leads to human-like syntactic knowledge. Furthermore, existing work has not provided a clear picture about the model properties required to produce proper syntactic generalizations. We present a systematic evaluation of the syntactic knowledge of neural language models, testing 20 combinations of model types and data sizes on a set of 34 English-language syntactic test suites. We find substantial differences in syntactic generalization performance by model architecture, with sequential models underperforming other architectures. Factorially manipulating model architecture and training dataset size (1M--40M words), we find that variability in syntactic generalization performance is substantially greater by architecture than by dataset size for the corpora tested in our experiments. Our results also reveal a dissociation between perplexity and syntactic generalization performance.

cs.CL↗

arXiv2020

Detecting dynamical quantum phase transition via out-of-time-order correlations in a solid-state quantum simulator

Quantum many body system in equilibrium can be effectively characterized using the framework of quantum statistical mechanics. However, nonequilibrium behaviour of quantum many body systems remains elusive, out of the range of such a well established framework. Experiments in quantum simulators are now opening up a route towards the generation of quantum states beyond this equilibrium paradigm. As an example in closed quantum many body systems, dynamical quantum phase transitions behave as phase transitions in time with physical quantities becoming nonanalytic at critical times, extending important principles such as universality to the nonequilibrium realm. Here, in solid state quantum simulator we develop and experimentally demonstrate that out-of-time-order correlators, a central concept to quantify quantum information scrambling and quantum chaos, can be used to dynamically detect nonoequilibrium phase transitions in the transverse field Ising model. We also study the multiple quantum spectra, eventually observe the buildup of quantum correlation. Further applications of this protocol could enable studies other of exotic phenomena such as many body localization, and tests of the holographic duality between quantum and gravitational systems.

quant-ph↗

arXiv2019

Representation of Constituents in Neural Language Models: Coordination Phrase as a Case Study

Neural language models have achieved state-of-the-art performances on many NLP tasks, and recently have been shown to learn a number of hierarchically-sensitive syntactic dependencies between individual words. However, equally important for language processing is the ability to combine words into phrasal constituents, and use constituent-level features to drive downstream expectations. Here we investigate neural models' ability to represent constituent-level features, using coordinated noun phrases as a case study. We assess whether different neural language models trained on English and French represent phrase-level number and gender features, and use those features to drive downstream expectations. Our results suggest that models use a linear combination of NP constituent number to drive CoordNP/verb number agreement. This behavior is highly regular and even sensitive to local syntactic context, however it differs crucially from observed human behavior. Models have less success with gender agreement. Models trained on large corpora perform best, and there is no obvious advantage for models trained using explicit syntactic supervision.

cs.CL↗

arXiv2019

Structural Supervision Improves Learning of Non-Local Grammatical Dependencies

State-of-the-art LSTM language models trained on large corpora learn sequential contingencies in impressive detail and have been shown to acquire a number of non-local grammatical dependencies with some success. Here we investigate whether supervision with hierarchical structure enhances learning of a range of grammatical dependencies, a question that has previously been addressed only for subject-verb agreement. Using controlled experimental methods from psycholinguistics, we compare the performance of word-based LSTM models versus two models that represent hierarchical structure and deploy it in left-to-right processing: Recurrent Neural Network Grammars (RNNGs) (Dyer et al., 2016) and a incrementalized version of the Parsing-as-Language-Modeling configuration from Chariak et al., (2016). Models are tested on a diverse range of configurations for two classes of non-local grammatical dependencies in English---Negative Polarity licensing and Filler--Gap Dependencies. Using the same training data across models, we find that structurally-supervised models outperform the LSTM, with the RNNG demonstrating best results on both types of grammatical dependencies and even learning many of the Island Constraints on the filler--gap dependency. Structural supervision thus provides data efficiency advantages over purely string-based training of neural language models in acquiring human-like generalizations about non-local grammatical dependencies.

cs.CL↗

arXiv2019

A quantum algorithm for solving systems of nonlinear algebraic equations

We propose a quantum algorithm to solve systems of nonlinear algebraic equations. In the ideal case the complexity of the algorithm is linear in the number of variables $n$, which means our algorithm's complexity is less than $O(n^{3})$ of the corresponding classical ones when the set of equations has many variables and not so high orders. Different with most popular way of representing the results by state vector, we get the results in computation basis which is readable directly.

quant-ph↗

arXiv2019

Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic State

We deploy the methods of controlled psycholinguistic experimentation to shed light on the extent to which the behavior of neural network language models reflects incremental representations of syntactic state. To do so, we examine model behavior on artificial sentences containing a variety of syntactically complex structures. We test four models: two publicly available LSTM sequence models of English (Jozefowicz et al., 2016; Gulordava et al., 2018) trained on large datasets; an RNNG (Dyer et al., 2016) trained on a small, parsed dataset; and an LSTM trained on the same small corpus as the RNNG. We find evidence that the LSTMs trained on large datasets represent syntactic state over large spans of text in a way that is comparable to the RNNG, while the LSTM trained on the small dataset does not or does so only weakly.

cs.CL↗

arXiv2016

Magnetogenesis in bouncing cosmology

We consider the process of magnetogenesis in the context of nonsingular bounce cosmology. We show that large primordial magnetic fields can be generated during contraction without encountering strong coupling and backreaction issues. The fields may seed large-scale magnetic fields with observationally interesting strengths. This result leads to a theoretical constraint on the relation of the energy scale of the bounce cosmology to the number of effective e-folding of the contracting phase in the case of scale invariance for the power spectrum of primordial magnetic fields. We show that this constraint can be satisfied in a sizable region of the parameter space for the nonsingular bounce cosmology.

gr-qc↗

arXiv2016

Bridging LSTM Architecture and the Neural Dynamics during Reading

Recently, the long short-term memory neural network (LSTM) has attracted wide interest due to its success in many tasks. LSTM architecture consists of a memory cell and three gates, which looks similar to the neuronal networks in the brain. However, there still lacks the evidence of the cognitive plausibility of LSTM architecture as well as its working mechanism. In this paper, we study the cognitive plausibility of LSTM by aligning its internal architecture with the brain activity observed via fMRI when the subjects read a story. Experiment results show that the artificial memory vector in LSTM can accurately predict the observed sequential brain activities, indicating the correlation between LSTM architecture and the cognitive process of story reading.

cs.CL↗

arXiv2016

Model of inflationary magnetogenesis

We consider the possibility of inflationary magnetogenesis due to dynamical couplings of the electromagnetic fields to gravity. We find that large primordial magnetic fields can be generated during inflation without the strong coupling problem, backreaction problem, or curvature perturbation problem, which seed large-scale magnetic fields with observationally interesting strengths.

astro-ph.CO↗

arXiv2015

Overview of the NLPCC 2015 Shared Task: Chinese Word Segmentation and POS Tagging for Micro-blog Texts

In this paper, we give an overview for the shared task at the 4th CCF Conference on Natural Language Processing \& Chinese Computing (NLPCC 2015): Chinese word segmentation and part-of-speech (POS) tagging for micro-blog texts. Different with the popular used newswire datasets, the dataset of this shared task consists of the relatively informal micro-texts. The shared task has two sub-tasks: (1) individual Chinese word segmentation and (2) joint Chinese word segmentation and POS Tagging. Each subtask has three tracks to distinguish the systems with different resources. We first introduce the dataset and task, then we characterize the different approaches of the participating systems, report the test results, and provide a overview analysis of these results. An online system is available for open registration and evaluation at http://nlp.fudan.edu.cn/nlpcc2015.

cs.CL↗

arXiv2014

Lifshitz Scaling Effects on Holographic Superconductors

Via numerical and analytical methods, the effects of the Lifshitz dynamical exponent $z$ on holographic superconductors are studied in some detail, including $s$ wave and $p$ wave models. Working in the probe limit, we find that the behaviors of holographic models indeed depend on concrete value of $z$. We obtain the condensation and conductivity in both Lifshitz black hole and soliton backgrounds with general $z$. For both $s$ wave and $p$ wave models in the black hole backgrounds, as $z$ increases, the phase transition becomes more difficult and the growth of conductivity is suppressed. For the Lifshitz soliton backgrounds, when $z$ increases ($z=1,~2,~3$), the critical chemical potential decreases in the $s$ wave cases but increases in the $p$ wave cases. For $p$ wave models in both Lifshitz black hole and soliton backgrounds, the anisotropy between the AC conductivity in different spatial directions is suppressed when $z$ increases. The analytical results uphold the numerical results.

hep-th↗