arXiv subjects

Jun Suzuki

Publications and source records attributed to Jun Suzuki.

At least 109 records · Page 6Linked to original sources

arXiv2019

An Empirical Study of Incorporating Pseudo Data into Grammatical Error Correction

The incorporation of pseudo data in the training of grammatical error correction models has been one of the main factors in improving the performance of such models. However, consensus is lacking on experimental configurations, namely, choosing how the pseudo data should be generated or used. In this study, these choices are investigated through extensive experiments, and state-of-the-art performance is achieved on the CoNLL-2014 test set ($F_{0.5}=65.0$) and the official test set of the BEA-2019 shared task ($F_{0.5}=70.2$) without making any modifications to the model architecture.

cs.CL↗

arXiv2019

Character n-gram Embeddings to Improve RNN Language Models

This paper proposes a novel Recurrent Neural Network (RNN) language model that takes advantage of character information. We focus on character n-grams based on research in the field of word embedding construction (Wieting et al. 2016). Our proposed method constructs word embeddings from character n-gram embeddings and combines them with ordinary word embeddings. We demonstrate that the proposed method achieves the best perplexities on the language modeling datasets: Penn Treebank, WikiText-2, and WikiText-103. Moreover, we conduct experiments on application tasks: machine translation and headline generation. The experimental results indicate that our proposed method also positively affects these tasks.

cs.CL↗

arXiv2019

Nuisance parameter problem in quantum estimation theory: General formulation and qubit examples

In this paper, we analyze quantum-state estimation problems when some of the parameters are of no interest to be estimated. In classical statistics, these irrelevant parameters are called nuisance parameters and this problem is of great importance in many practical applications of statistics. However, little is known regarding the effects of nuisance parameters in quantum-state estimation problems. The main contribution of this paper is first to formulate the nuisance parameter problem for the quantum-state estimation theory, then to propose a method of how to eliminate the nuisance parameters to obtain an estimation error bound for the parameters of interest. We also develop useful methods of dealing with the nuisance parameter problem in the quantum case and reveal the significant difference from the classical case. In particular, we clarify an intrinsic tradeoff relation to estimate the nuisance parameters and parameters of interest. The general qubit model is examined in detail to emphasize that we cannot ignore the effects of the nuisance parameters in general. Several examples in qubit systems are worked out to illustrate our findings.

quant-ph↗

arXiv2019

Quantum process tomography via optimal design of experiments

Quantum process tomography --- a primitive in many quantum information processing tasks --- can be cast within the framework of the theory of design of experiment (DoE), a branch of classical statistics that deals with the relationship between inputs and outputs of an experimental setup. Such a link potentially gives access to the many ideas of the rich subject of classical DoE for use in quantum problems. The classical techniques from DoE cannot, however, be directly applied to the quantum process tomography due to the basic structural differences between the classical and quantum estimation problems. Here, we properly formulate quantum process tomography as a DoE problem, and examine several examples to illustrate the link and the methods. In particular, we discuss the common issue of nuisance parameters, and point out interesting features in the quantum problem absent in the usual classical setting.

quant-ph↗

arXiv2019

Personalized Visited-POI Assignment to Individual Raw GPS Trajectories

Knowledge discovery from GPS trajectory data is an important topic in several scientific areas, including data mining, human behavior analysis, and user modeling. This paper proposes a task that assigns personalized visited-POIs. Its goal is to estimate fine-grained and pre-defined locations (i.e., points of interest (POI)) that are actually visited by users and assign visited-location information to the corresponding span of their (personal) GPS trajectories. We also introduce a novel algorithm to solve this assignment task. First, we exhaustively extract stay-points as candidates for significant locations using a variant of a conventional stay-point extraction method. Then we select significant locations and simultaneously assign visited-POIs to them by considering various aspects, which we formulate in integer linear programming. Experimental results conducted on an actual user dataset show that our method achieves higher accuracy in the visited-POI assignment task than the various cascaded procedures of conventional methods.

cs.CY↗

arXiv2018

Mixture of Expert/Imitator Networks: Scalable Semi-supervised Learning Framework

The current success of deep neural networks (DNNs) in an increasingly broad range of tasks involving artificial intelligence strongly depends on the quality and quantity of labeled training data. In general, the scarcity of labeled data, which is often observed in many natural language processing tasks, is one of the most important issues to be addressed. Semi-supervised learning (SSL) is a promising approach to overcoming this issue by incorporating a large amount of unlabeled data. In this paper, we propose a novel scalable method of SSL for text classification tasks. The unique property of our method, Mixture of Expert/Imitator Networks, is that imitator networks learn to "imitate" the estimated label distribution of the expert network over the unlabeled data, which potentially contributes a set of features for the classification. Our experiments demonstrate that the proposed method consistently improves the performance of several types of baseline DNNs. We also demonstrate that our method has the more data, better performance property with promising scalability to the amount of unlabeled data.

cs.CL↗

arXiv2018

Pointwise HSIC: A Linear-Time Kernelized Co-occurrence Norm for Sparse Linguistic Expressions

In this paper, we propose a new kernel-based co-occurrence measure that can be applied to sparse linguistic expressions (e.g., sentences) with a very short learning time, as an alternative to pointwise mutual information (PMI). As well as deriving PMI from mutual information, we derive this new measure from the Hilbert--Schmidt independence criterion (HSIC); thus, we call the new measure the pointwise HSIC (PHSIC). PHSIC can be interpreted as a smoothed variant of PMI that allows various similarity metrics (e.g., sentence embeddings) to be plugged in as kernels. Moreover, PHSIC can be estimated by simple and fast (linear in the size of the data) matrix calculations regardless of whether we use linear or nonlinear kernels. Empirically, in a dialogue response selection task, PHSIC is learned thousands of times faster than an RNN-based PMI while outperforming PMI in accuracy. In addition, we also demonstrate that PHSIC is beneficial as a criterion of a data selection task for machine translation owing to its ability to give high (low) scores to a consistent (inconsistent) pair with other pairs.

cs.CL↗

arXiv2018

Direct Output Connection for a High-Rank Language Model

This paper proposes a state-of-the-art recurrent neural network (RNN) language model that combines probability distributions computed not only from a final RNN layer but also from middle layers. Our proposed method raises the expressive power of a language model based on the matrix factorization interpretation of language modeling introduced by Yang et al. (2018). The proposed method improves the current state-of-the-art language model and achieves the best score on the Penn Treebank and WikiText-2, which are the standard benchmark datasets. Moreover, we indicate our proposed method contributes to two application tasks: machine translation and headline generation. Our code is publicly available at: https://github.com/nttcslab-nlp/doc_lm.

cs.CL↗

arXiv2018

Classification and characterization of quantum parametric models in quantum estimation theory

In this paper, we characterize quantum parametric models into different classes based on the estimation error bound, known as the Holevo bound. These classes are given by the classical, quasi-classical, D-invariant, and asymptotically classical models. We first explore the relationships among these four models and show that: i) The classical model having the diagonal elements only is characterized by the intersection of the D-invariant and asymptotically classical models. ii) There exists a gap between the classical model and the quasi-classical model, where all logarithmic derivative operators commute with each other. Further, we characterize each class with several equivalent conditions. This result then reveals the geometrical understanding of quantum statistical models.

quant-ph↗

arXiv2018

Interpretable Adversarial Perturbation in Input Embedding Space for Text

Following great success in the image processing field, the idea of adversarial training has been applied to tasks in the natural language processing (NLP) field. One promising approach directly applies adversarial training developed in the image processing field to the input word embedding space instead of the discrete input space of texts. However, this approach abandons such interpretability as generating adversarial texts to significantly improve the performance of NLP tasks. This paper restores interpretability to such methods by restricting the directions of perturbations toward the existing words in the input embedding space. As a result, we can straightforwardly reconstruct each input with perturbations to an actual text by considering the perturbations to be the replacement of words in the sentence while maintaining or even improving the task performance.

cs.LG↗

arXiv2017

Source-side Prediction for Neural Headline Generation

The encoder-decoder model is widely used in natural language generation tasks. However, the model sometimes suffers from repeated redundant generation, misses important phrases, and includes irrelevant entities. Toward solving these problems we propose a novel source-side token prediction module. Our method jointly estimates the probability distributions over source and target vocabularies to capture a correspondence between source and target tokens. The experiments show that the proposed model outperforms the current state-of-the-art method in the headline generation task. Additionally, we show that our method has an ability to learn a reasonable token-wise correspondence without knowing any true alignments.

cs.CL↗

arXiv2017

Input-to-Output Gate to Improve RNN Language Models

This paper proposes a reinforcing method that refines the output layers of existing Recurrent Neural Network (RNN) language models. We refer to our proposed method as Input-to-Output Gate (IOG). IOG has an extremely simple structure, and thus, can be easily combined with any RNN language models. Our experiments on the Penn Treebank and WikiText-2 datasets demonstrate that IOG consistently boosts the performance of several different types of current topline RNN language models.

cs.CL↗

arXiv2017

Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization

This paper tackles the reduction of redundant repeating generation that is often observed in RNN-based encoder-decoder models. Our basic idea is to jointly estimate the upper-bound frequency of each target vocabulary in the encoder and control the output words based on the estimation in the decoder. Our method shows significant improvement over a strong RNN-based encoder-decoder baseline and achieved its best results on an abstractive summarization benchmark.

cs.CL↗

arXiv2017

Enumeration of Extractive Oracle Summaries

To analyze the limitations and the future directions of the extractive summarization paradigm, this paper proposes an Integer Linear Programming (ILP) formulation to obtain extractive oracle summaries in terms of ROUGE-N. We also propose an algorithm that enumerates all of the oracle summaries for a set of reference summaries to exploit F-measures that evaluate which system summaries contain how many sentences that are extracted as an oracle summary. Our experimental results obtained from Document Understanding Conference (DUC) corpora demonstrated the following: (1) room still exists to improve the performance of extractive summarization; (2) the F-measures derived from the enumerated oracle summaries have significantly stronger correlations with human judgment than those derived from single oracle summaries.

cs.CL↗

arXiv2016

Explicit formula for the Holevo bound for two-parameter qubit estimation problem

The main contribution of this paper is to derive an explicit expression for the fundamental precision bound, the Holevo bound, for estimating any two-parameter family of qubit mixed-states in terms of quantum versions of Fisher information. The obtained formula depends solely on the symmetric logarithmic derivative (SLD), the right logarithmic derivative (RLD) Fisher information, and a given weight matrix. This result immediately provides necessary and sufficient conditions for the following two important classes of quantum statistical models; the Holevo bound coincides with the SLD Cramer-Rao bound and it does with the RLD Cramer-Rao bound. One of the important results of this paper is that a general model other than these two special cases exhibits an unexpected property: The structure of the Holevo bound changes smoothly when the weight matrix varies. In particular, it always coincides with the RLD Cramer-Rao bound for a certain choice of the weight matrix. Several examples illustrate these findings.

quant-ph↗

arXiv2015

Entanglement detection from channel parameter estimation problem

We derive a general criterion to detect entangled states in multi-partite systems based on the symmetric logarithmic derivative quantum Fisher information. This criterion is a direct consequence of the fact that separable states do not improve the accuracy upon estimating one-parameter family of quantum channels. Our result is a generalization of the previously known criterion for one-parameter unitary channel to any one-parameter quantum channel. We discuss several examples to illustrate our criterion. The proposed criterion is extended to the case of open quantum systems and we briefly discuss how to detect entangled states in the presence of decoherence.

quant-ph↗

arXiv2015

Parameter estimation of qubit states with unknown phase parameter

We discuss a problem of parameter estimation for quantum two-level system, qubit system, in presence of unknown phase parameter. We analyze trade-off relations for mean-square errors when estimating relevant parameters with separable measurements based on known precision bounds; the symmetric logarithmic derivative Cramer-Rao bound and Hayashi-Gill-Massar (HGM) bound. We investigate the optimal measurement which attains the HGM bound and discuss its properties. We show that the HGM bound for relevant parameters can be attained asymptotically by using some fraction of given $n$ quantum states to estimate the phase parameter. We also discuss the Holevo bound which can be attained asymptotically by a collective measurement.

quant-ph↗

arXiv2014

Qubit subalgebra and tensor product in Weyl algebra of angular momentum system

We analyze Weyl algebra of quantum angular momentum system and construct qubit subalgebra out of it. We show that the commutant of this qubit subalgebra is isomorphic to the original algebra and prove the tensor product structure between qubit subalgebra and its commutant. This construction can be iterated to construct arbitrary number of qubit subalgebras from a single quantum system. We show a simple experimental realization of this proposed scheme using orbital angular momentum of single photons. We briefly discuss about construction of qudit subalgbra and generalization to other infinite dimensional systems.

quant-ph↗