arXiv subjects

Miguel Ballesteros

Publications and source records attributed to Miguel Ballesteros.

At least 73 records · Page 4Linked to original sources

arXiv2019

Analyticity of Resonances and Eigenvalues and Spectral Properties of the massless Spin-Boson Model

We extend the method of multiscale analysis for resonances introduced in [5] in order to infer analytic properties of resonances and eigenvalues (and their eigenprojections) as well as estimates for the localization of the spectrum of dilated Hamiltonians and norm-bounds for the corresponding resolvent operators, in neighborhoods of resonances and eigenvalues. We apply our method to the massless Spin-Boson model assuming a slight infrared regularization. We prove that the resonance and the ground-state eigenvalue (and their eigenprojections) are analytic with respect to the dilation parameter and the coupling constant. Moreover, we prove that the spectrum of the dilated Spin-Boson Hamiltonian in the neighborhood of the resonance and the ground-state eigenvalue is localized in two cones in the complex plane with vertices at the location of the resonance and the ground-state eigenvalue, respectively. Additionally, we provide norm-estimates for the resolvent of the dilated Spin-Boson Hamiltonian near the resonance and the ground-state eigenvalue. The topic of analyticity of eigenvalues and resonances has let to several studies and advances in the past. However, to the best of our knowledge, this is the first time that it is addressed from the perspective of multiscale analysis. Once the multiscale analysis is set up our method gives easy access to analyticity: Essentially, it amounts to proving it for isolated eigenvalues only and use that uniform limits of analytic functions are analytic. The type of spectral and resolvent estimates that we prove are needed to control the time evolution including the scattering regime. The latter will be demonstrated in a forthcoming publication. The introduced multiscale method to study spectral and resolvent estimates follows its own inductive scheme and is independent (and different) from the method we apply to construct resonances.

math-ph↗

arXiv2019

Conditionally Free Reduced Products of Hilbert Spaces

We present a product of pairs of pointed Hilbert spaces that, in the context of Bozėjko, Leinert and Speicher's theory of conditionally free probability, plays the role of the reduced free product of pointed Hilbert spaces, and thus gives a unified construction for the natural notions of independence defined by Muraki. We additionally provide important applications of this construction. We prove that, assuming minor restrictions, for any pair of conditionally free algebras there are copies of them that are conditionally free and also free, a property that is frequently assumed (as hypothesis) to prove several results in the literature. Finally, we give a short proof of the linearization property of the $^cR$-transform (the analog of Voiculescu's $R$-transform in the context of conditionally free probability).

math.OA↗

arXiv2018

Greedy, Joint Syntactic-Semantic Parsing with Stack LSTMs

We present a transition-based parser that jointly produces syntactic and semantic dependencies. It learns a representation of the entire algorithm state, using stack long short-term memories. Our greedy inference algorithm has linear time, including feature extraction. On the CoNLL 2008--9 English shared tasks, we obtain the best published parsing performance among models that jointly learn syntax and semantics.

cs.CL↗

arXiv2018

Multilingual Neural Machine Translation with Task-Specific Attention

Multilingual machine translation addresses the task of translating between multiple source and target languages. We propose task-specific attention models, a simple but effective technique for improving the quality of sequence-to-sequence neural multilingual translation. Our approach seeks to retain as much of the parameter sharing generalization of NMT models as possible, while still allowing for language-specific specialization of the attention model to a particular language-pair or task. Our experiments on four languages of the Europarl corpus show that using a target-specific model of attention provides consistent gains in translation quality for all possible translation directions, compared to a model in which all parameters are shared. We observe improved translation quality even in the (extreme) low-resource zero-shot translation directions for which the model never saw explicitly paired parallel data.

cs.CL↗

arXiv2018

Scheduled Multi-Task Learning: From Syntax to Translation

Neural encoder-decoder models of machine translation have achieved impressive results, while learning linguistic knowledge of both the source and target languages in an implicit end-to-end manner. We propose a framework in which our model begins learning syntax and translation interleaved, gradually putting more focus on translation. Using this approach, we achieve considerable improvements in terms of BLEU score on relatively large parallel corpus (WMT14 English to German) and a low-resource (WIT German to English) setup.

cs.CL↗

arXiv2018

Multimodal Emoji Prediction

Emojis are small images that are commonly included in social media text messages. The combination of visual and textual content in the same message builds up a modern way of communication, that automatic systems are not used to deal with. In this paper we extend recent advances in emoji prediction by putting forward a multimodal approach that is able to predict emojis in Instagram posts. Instagram posts are composed of pictures together with texts which sometimes include emojis. We show that these emojis can be predicted by using the text, but also using the picture. Our main finding is that incorporating the two synergistic modalities, in a combined model, improves accuracy in an emoji prediction task. This result demonstrates that these two modalities (text and images) encode different information on the use of emojis and therefore can complement each other.

cs.CL↗

arXiv2018

Pieces of Eight: 8-bit Neural Machine Translation

Neural machine translation has achieved levels of fluency and adequacy that would have been surprising a short time ago. Output quality is extremely relevant for industry purposes, however it is equally important to produce results in the shortest time possible, mainly for latency-sensitive applications and to control cloud hosting costs. In this paper we show the effectiveness of translating with 8-bit quantization for models that have been trained using 32-bit floating point values. Results show that 8-bit translation makes a non-negligible impact in terms of speed with no degradation in accuracy and adequacy.

cs.CL↗

arXiv2017

Arc-Standard Spinal Parsing with Stack-LSTMs

We present a neural transition-based parser for spinal trees, a dependency representation of constituent trees. The parser uses Stack-LSTMs that compose constituent nodes with dependency-based derivations. In experiments, we show that this model adapts to different styles of dependency relations, but this choice has little effect for predicting constituent structure, suggesting that LSTMs induce useful states by themselves.

cs.CL↗

arXiv2017

AMR Parsing using Stack-LSTMs

We present a transition-based AMR parser that directly generates AMR parses from plain text. We use Stack-LSTMs to represent our parser state and make decisions greedily. In our experiments, we show that our parser achieves very competitive scores on English using only AMR training data. Adding additional information, such as POS tags and dependency trees, improves the results further.

cs.CL↗

arXiv2017

Are Emojis Predictable?

Emojis are ideograms which are naturally combined with plain text to visually complement or condense the meaning of a message. Despite being widely used in social media, their underlying semantics have received little attention from a Natural Language Processing standpoint. In this paper, we investigate the relation between words and emojis, studying the novel task of predicting which emojis are evoked by text-based tweet messages. We train several models based on Long Short-Term Memory networks (LSTMs) in this task. Our experimental results show that our neural model outperforms two baselines as well as humans solving the same task, suggesting that computational models are able to better capture the underlying semantics of emojis.

cs.CL↗

arXiv2017

DyNet: The Dynamic Neural Network Toolkit

We describe DyNet, a toolkit for implementing neural network models based on dynamic declaration of network structure. In the static declaration strategy that is used in toolkits like Theano, CNTK, and TensorFlow, the user first defines a computation graph (a symbolic representation of the computation), and then examples are fed into an engine that executes this computation and computes its derivatives. In DyNet's dynamic declaration strategy, computation graph construction is mostly transparent, being implicitly constructed by executing procedural code that computes the network outputs, and the user is free to use different network structures for each input. Dynamic declaration thus facilitates the implementation of more complicated network architectures, and DyNet is specifically designed to allow users to implement their models in a way that is idiomatic in their preferred programming language (C++ or Python). One challenge with dynamic declaration is that because the symbolic computation graph is defined anew for every training example, its construction must have low overhead. To achieve this, DyNet has an optimized C++ backend and lightweight graph representation. Experiments show that DyNet's speeds are faster than or comparable with static declaration toolkits, and significantly faster than Chainer, another dynamic declaration toolkit. DyNet is released open-source under the Apache 2.0 license and available at http://github.com/clab/dynet.

stat.ML↗

arXiv2017

What Do Recurrent Neural Network Grammars Learn About Syntax?

Recurrent neural network grammars (RNNG) are a recently proposed probabilistic generative modeling family for natural language. They show state-of-the-art language modeling and parsing performance. We investigate what information they learn, from a linguistic perspective, through various ablations to the model and the data, and by augmenting the model with an attention mechanism (GA-RNNG) to enable closer inspection. We find that explicit modeling of composition is crucial for achieving the best performance. Through the attention mechanism, we find that headedness plays a central role in phrasal representation (with the model's latent attention largely agreeing with predictions made by hand-crafted head rules, albeit with some important differences). By training grammars without nonterminal labels, we find that phrasal representations depend minimally on nonterminals, providing support for the endocentricity hypothesis.

cs.CL↗

arXiv2016

Recurrent Neural Network Grammars

We introduce recurrent neural network grammars, probabilistic models of sentences with explicit phrase structure. We explain efficient inference procedures that allow application to both parsing and language modeling. Experiments show that they provide better parsing in English than any single previously published supervised generative model and better language modeling than state-of-the-art sequential RNNs in English and Chinese.

cs.CL↗

arXiv2016

Distilling an Ensemble of Greedy Dependency Parsers into One MST Parser

We introduce two first-order graph-based dependency parsers achieving a new state of the art. The first is a consensus parser built from an ensemble of independently trained greedy LSTM transition-based parsers with different random initializations. We cast this approach as minimum Bayes risk decoding (under the Hamming cost) and argue that weaker consensus within the ensemble is a useful signal of difficulty or ambiguity. The second parser is a "distillation" of the ensemble into a single model. We train the distillation parser using a structured hinge loss objective with a novel cost that incorporates ensemble uncertainty estimates for each possible attachment, thereby avoiding the intractable cross-entropy computations required by applying standard distillation objectives to problems with structured outputs. The first-order distillation parser matches or surpasses the state of the art on English, Chinese, and German.

cs.CL↗

arXiv2016

Training with Exploration Improves a Greedy Stack-LSTM Parser

We adapt the greedy Stack-LSTM dependency parser of Dyer et al. (2015) to support a training-with-exploration procedure using dynamic oracles(Goldberg and Nivre, 2013) instead of cross-entropy minimization. This form of training, which accounts for model predictions at training time rather than assuming an error-free action history, improves parsing accuracies for both English and Chinese, obtaining very strong results for both languages. We discuss some modifications needed in order to get training with exploration to work well for a probabilistic neural-network.

cs.CL↗

arXiv2016

Many Languages, One Parser

We train one multilingual model for dependency parsing and use it to parse sentences in several languages. The parsing model uses (i) multilingual word clusters and embeddings; (ii) token-level language information; and (iii) language-specific features (fine-grained POS tags). This input representation enables the parser not only to parse effectively in multiple languages, but also to generalize across languages based on linguistic universals and typological similarities, making it more effective to learn from limited annotations. Our parser's performance compares favorably to strong baselines in a range of data scenarios, including when the target language has a large treebank, a small treebank, or no treebank for training.

cs.CL↗

arXiv2016

Existence of Ground State Eigenvalues for the Spin-Boson Model with Critical Infrared Divergence and Multiscale Analysis

A two-level atom coupled to the radiation field is studied. First principles in physics suggest that the coupling function, representing the interaction between the atom and the radiation field, behaves like $\vert k \vert^{- 1/2}$, as the photon momentum k tends to zero. Previous results on non-existence of ground state eigenvalues suggest that in the most general case binding does not occur in the spin-boson model, i.e., the minimal energy of the atom-photon system is not an eigenvalue of the energy operator. Hasler and Herbst have shown [12], however, that under the additional hypothesis that the coupling function be off-diagonal -which is customary to assume-binding does indeed occur. In this paper an alternative proof of binding in case of off-diagonal coupling is given, i.e., it is proven that, if the coupling function is off-diagonal, the ground state energy of the spin-boson model is an eigenvalue of the Hamiltonian. We develop a multiscale method that can be applied in the situation we study, identifying a new key symmetry operator which we use to demonstrate that the most singular terms appearing in the multiscale analysis vanish.

math-ph↗

arXiv2016

Neural Architectures for Named Entity Recognition

State-of-the-art named entity recognition systems rely heavily on hand-crafted features and domain-specific knowledge in order to learn effectively from the small, supervised training corpora that are available. In this paper, we introduce two new neural architectures---one based on bidirectional LSTMs and conditional random fields, and the other that constructs and labels segments using a transition-based approach inspired by shift-reduce parsers. Our models rely on two sources of information about words: character-based word representations learned from the supervised corpus and unsupervised word representations learned from unannotated corpora. Our models obtain state-of-the-art performance in NER in four languages without resorting to any language-specific knowledge or resources such as gazetteers.

cs.CL↗