arXiv subjects

Jiajun Zhang

Publications and source records attributed to Jiajun Zhang.

At least 235 records · Page 13Linked to original sources

arXiv2019

End-to-End Speech Translation with Knowledge Distillation

End-to-end speech translation (ST), which directly translates from source language speech into target language text, has attracted intensive attentions in recent years. Compared to conventional pipeline systems, end-to-end ST models have advantages of lower latency, smaller model size and less error propagation. However, the combination of speech recognition and text translation in one model is more difficult than each of these two tasks. In this paper, we propose a knowledge distillation approach to improve ST model by transferring the knowledge from text translation model. Specifically, we first train a text translation model, regarded as a teacher model, and then ST model is trained to learn output probabilities from teacher model through knowledge distillation. Experiments on English- French Augmented LibriSpeech and English-Chinese TED corpus show that end-to-end ST is possible to implement on both similar and dissimilar language pairs. In addition, with the instruction of teacher model, end-to-end ST model can gain significant improvements by over 3.5 BLEU points.

cs.CL↗

arXiv2019

The first constraint from SDSS galaxy-galaxy weak lensing measurements on interacting dark energy models

We combine constraints from linear and nonlinear scales, for the first time, to study the interaction between dark matter and dark energy. We devise a novel N-body simulation pipeline for cosmological models beyond $Λ$CDM. This pipeline is fully self-consistent and opens a new window to study the nonlinear structure formation in general phenomenological interacting dark energy models. By comparing our simulation results with the SDSS galaxy-galaxy weak lensing measurements, we are able to constrain the strength of interaction between dark energy and dark matter. Compared with the previous studies using linear examinations, we point to plausible improvements on the constraints of interaction strength by using small scale information from weak lensing. This improvement is mostly due to the sensitivity of weak lensing measurements on nonlinear structure formation at low redshift. With this new pipeline, it is possible to look for smoking gun signatures of dark matter-dark energy interaction.

astro-ph.CO↗

arXiv2019

Constraining Dark Energy With Stacked Concave Lenses

Low density regions are less affected by the nonlinear structure formation and baryonic physics. They are ideal places for probing the nature of dark energy, a possible explanation for the cosmic acceleration. Unlike void lensing, which requires identifications of individual voids, we study the stacked lensing signals around the low-density-positions (LDP), defined as places that are devoid of foreground bright galaxies in projection. The method allows a direct comparison with numerical results by drawing correspondence between the bright galaxies with halos. It leads to lensing signals that are significant enough for differentiating several dark energy models. In this work, we use the CFHTLenS catalogue to define LDPs, as well as measuring their background lensing signals. We consider several different definitions of the foreground bright galaxies (redshift range \& magnitude cut). Regarding the cosmological model, we run six simulations: the first set of simulations have the same initial conditions, with $\rm{w_{de}=-1,-0.5,-0.8,-1.2}$; the second set of simulations include a slightly different $Λ$CDM model and a w(z) model from \cite{2017NatAs...1..627Z}. The lensing results indicate that the models with $\rm{w_{de}=-0.5,-0.8}$ are not favored, and the other four models all achieve comparable agreement with the data.

astro-ph.CO↗

arXiv2019

Synchronous Bidirectional Inference for Neural Sequence Generation

In sequence to sequence generation tasks (e.g. machine translation and abstractive summarization), inference is generally performed in a left-to-right manner to produce the result token by token. The neural approaches, such as LSTM and self-attention networks, are now able to make full use of all the predicted history hypotheses from left side during inference, but cannot meanwhile access any future (right side) information and usually generate unbalanced outputs in which left parts are much more accurate than right ones. In this work, we propose a synchronous bidirectional inference model to generate outputs using both left-to-right and right-to-left decoding simultaneously and interactively. First, we introduce a novel beam search algorithm that facilitates synchronous bidirectional decoding. Then, we present the core approach which enables left-to-right and right-to-left decoding to interact with each other, so as to utilize both the history and future predictions simultaneously during inference. We apply the proposed model to both LSTM and self-attention networks. In addition, we propose two strategies for parameter optimization. The extensive experiments on machine translation and abstractive summarization demonstrate that our synchronous bidirectional inference model can achieve remarkable improvements over the strong baselines.

cs.CL↗

arXiv2019

Cosmological Simulation for Fuzzy Dark Matter Model

Fuzzy Dark Matter (FDM), motivated by string theory, has recently become a hot candidate for dark matter. The rest mass of FDM is believed to be $\sim 10^{-22}$eV and the corresponding de-Broglie wave length is $\sim 1$kpc. Therefore, the quantum effect of FDM plays an important role in structure formation. In order to study the cosmological structure formation in FDM model, several simulation techniques have been introduced. We review the current status and challenges in the cosmological simulation for the FDM model in this paper.

astro-ph.CO↗

arXiv2018

A Fully Self-Consistent Cosmological Simulation Pipeline for Interacting Dark Energy Models

We devise a fully self-consistent simulation pipeline for the first time to study the interaction between dark matter and dark energy. We perform convergence tests and show that our code is accurate on different scales. Using the parameters constrained by Planck, Type Ia Supernovae, Baryon Acoustic Oscillations (BAO) and Hubble constant observations, we perform cosmological N-body simulations. We calculate the resulting matter power spectra and halo mass functions for four different interacting dark energy models. In addition to the dark matter density distribution, we also show the inhomogeneous density distribution of dark energy. With this new simulation pipeline, we can further refine and constrain interacting dark energy models.

astro-ph.CO↗

arXiv2018

Language-Independent Representor for Neural Machine Translation

Current Neural Machine Translation (NMT) employs a language-specific encoder to represent the source sentence and adopts a language-specific decoder to generate target translation. This language-dependent design leads to large-scale network parameters and makes the duality of the parallel data underutilized. To address the problem, we propose in this paper a language-independent representor to replace the encoder and decoder by using weight sharing. This shared representor can not only reduce large portion of network parameters, but also facilitate us to fully explore the language duality by jointly training source-to-target, target-to-source, left-to-right and right-to-left translations within a multi-task learning framework. Experiments show that our proposed framework can obtain significant improvements over conventional NMT models on resource-rich and low-resource translation tasks with only a quarter of parameters.

cs.CL↗

arXiv2018

Source-Critical Reinforcement Learning for Transferring Spoken Language Understanding to a New Language

To deploy a spoken language understanding (SLU) model to a new language, language transferring is desired to avoid the trouble of acquiring and labeling a new big SLU corpus. Translating the original SLU corpus into the target language is an attractive strategy. However, SLU corpora consist of plenty of semantic labels (slots), which general-purpose translators cannot handle well, not to mention additional culture differences. This paper focuses on the language transferring task given a tiny in-domain parallel SLU corpus. The in-domain parallel corpus can be used as the first adaptation on the general translator. But more importantly, we show how to use reinforcement learning (RL) to further finetune the adapted translator, where translated sentences with more proper slot tags receive higher rewards. We evaluate our approach on Chinese to English language transferring for SLU systems. The experimental results show that the generated English SLU corpus via adaptation and reinforcement learning gives us over 97% in the slot F1 score and over 84% accuracy in domain classification. It demonstrates the effectiveness of the proposed language transferring method. Compared with naive translation, our proposed method improves domain classification accuracy by relatively 22%, and the slot filling F1 score by relatively more than 71%.

cs.CL↗

arXiv2018

The Importance of Quantum Pressure of Fuzzy Dark Matter on Lyman-Alpha Forest

With recent Lyman-alpha forest data from BOSS and XQ-100, some studies suggested that the lower mass limit on the fuzzy dark matter (FDM) particles is lifted up to $10^{-21}\,\mathrm{eV}$. However, such a limit was obtained by $Λ$CDM simulations with the FDM initial condition and the quantum pressure (QP) was not taken into account which could have generated non-trivial effects in large scales structures. We investigate the QP effects in cosmological simulations systematically, and find that the QP leads to further suppression of the matter power spectrum at small scales, as well as the halo mass function in the low mass end. We estimate the suppressing effect of QP in the 1D flux power spectrum of Lyman-alpha forest and compare it with data from BOSS and XQ-100. The rough uncertainties of thermal gas properties in the flux power spectrum model calculation were discussed. We conclude that more systematic studies, especially with QP taken into account, are necessary to constrain FDM particle mass using Lyman-alpha forest.

astro-ph.CO↗

arXiv2018

Phrase Table as Recommendation Memory for Neural Machine Translation

Neural Machine Translation (NMT) has drawn much attention due to its promising translation performance recently. However, several studies indicate that NMT often generates fluent but unfaithful translations. In this paper, we propose a method to alleviate this problem by using a phrase table as recommendation memory. The main idea is to add bonus to words worthy of recommendation, so that NMT can make correct predictions. Specifically, we first derive a prefix tree to accommodate all the candidate target phrases by searching the phrase translation table according to the source sentence. Then, we construct a recommendation word set by matching between candidate target phrases and previously translated target words by NMT. After that, we determine the specific bonus value for each recommendable word by using the attention vector and phrase translation probability. Finally, we integrate this bonus value into NMT to improve the translation results. The extensive experiments demonstrate that the proposed methods obtain remarkable improvements over the strong attentionbased NMT.

cs.CL↗

arXiv2018

New leaves of the tree: percolation analysis for cosmic web with discrete points

Percolation analysis has long been used to quantify the connectivity of the cosmic web. Most of the previous work is based on density fields on grids. By smoothing into fields, we lose information about galaxy properties like shape or luminosity. Lack of mathematical model also limits our understanding of percolation analysis. In order to overcome these difficulties, we have studied percolation analysis based on discrete points. Using a Friends-of-Friends (FoF) algorithm, we generate the S-bb relation, between the fractional mass of the largest connected group (S) and the FoF linking length (bb). We propose a new model, the Probability Cloud Cluster Expansion Theory (PCCET) to relate the S-bb relation with correlation functions. We show that the S-bb relation reflects a combination of all orders of correlation functions. Using N-body simulation, we find that the S-bb relation is robust against redshift distortion and incompleteness in observation. From the Bolshoi simulation, with Halo Abundance Matching (HAM), we have generated a mock galaxy catalogue. Good matching of the projected two-point correlation function with observation is confirmed. However, comparing the mock catalogue with the latest galaxy catalogue from SDSS DR12, we have found significant differences in their S-bb relations. This indicates that the mock galaxy catalogue cannot accurately retain higher order correlation functions than the two-point correlation function, which reveals the limit of HAM method. As a new measurement, S-bb relation is applicable to a wide range of data types, fast to compute, robust against redshift distortion and incompleteness, and it contains information of all orders of correlation function.

astro-ph.CO↗

arXiv2018

Ultra-Light Axion Dark Matter and its impacts on dark halo structure in $N$-body simulation

The Ultra-Light Axion (ULA) is a dark matter candidate with mass $\mathcal{O}(10^{-22})$eV and de Broglie wavelength of order kpc. Such an axion, also called the Fuzzy Dark Matter (FDM), thermalizes via the gravitational force and forms a Bose-Einstein condensate. Recent studies suggested that the quantum pressure from the FDM can significantly affect the structure formation in small scales, thus alleviating the so-called "small-scale crisis". In this paper, we develop a new technique to discretize the quantum pressure and illustrate the interactions among FDM particles in the $N$-body simulation, which accurately simulates the formation of the dark-matter halo and its inner structure in the region outside the softening length. In a self-gravitationally-bound virialized halo, we find a constant density, solitonic core, which is consistent with the theoretical prediction. The existence of the solitonic core reveals the non-linear effect of quantum pressure and impacts the structure formation in the FDM model.

astro-ph.CO↗

arXiv2018

Learning Multimodal Word Representation via Dynamic Fusion Methods

Multimodal models have been proven to outperform text-based models on learning semantic word representations. Almost all previous multimodal models typically treat the representations from different modalities equally. However, it is obvious that information from different modalities contributes differently to the meaning of words. This motivates us to build a multimodal model that can dynamically fuse the semantic representations from different modalities according to different types of words. To that end, we propose three novel dynamic fusion methods to assign importance weights to each modality, in which weights are learned under the weak supervision of word association pairs. The extensive experiments have demonstrated that the proposed methods outperform strong unimodal baselines and state-of-the-art multimodal models.

cs.CL↗

arXiv2017

An accurate centroid algorithm for PSF reconstruction

In this work, we present a novel centroiding method based on Fourier space Phase Fitting(FPF) for Point Spread Function(PSF) reconstruction. We generate two sets of simulations to test our method. The first set is generated by GalSim with elliptical Moffat profile and strong anisotropy which shifts the center of the PSF. The second set of simulation is drawn from CFHT i band stellar imaging data. We find non-negligible anisotropy from CFHT stellar images, which leads to $\sim$0.08 scatter in unit of pixels using polynomial fitting method Vakili and Hogg (2016). And we apply FPF method to estimate the centroid in real space, this scatter reduces to $\sim$0.04 in SNR=200 CFHT like sample. In low SNR (50 and 100) CFHT like samples, the background noise dominates the shifting of the centroid, therefore the scatter estimated from different methods are similar. We compare polynomial fitting and FPF using GalSim simulation with optical anisotropy. We find that in all SNR$\sim$50, 100 and 200) samples, FPF performs better than polynomial fitting by a factor of $\sim$3. In general, we suggest that in real observations there are anisotropy which shift the centroid, and FPF method is a better way to accurately locate it.

astro-ph.IM↗

arXiv2017

Galaxy-galaxy weak-lensing measurement from SDSS: II. host halo properties of galaxy groups

As the second paper of a series on studying galaxy-galaxy lensing signals using the Sloan Digital Sky Survey Data Release 7 (SDSS DR7), we present our measurement and modelling of the lensing signals around groups of galaxies. We divide the groups into four halo mass bins, and measure the signals around four different halo-center tracers: brightest central galaxy (BCG), luminosity-weighted center, number-weighted center and X-ray peak position. For X-ray and SDSS DR7 cross identified groups, we further split the groups into low and high X-ray emission subsamples, both of which are assigned with two halo-center tracers, BCGs and X-ray peak positions. The galaxy-galaxy lensing signals show that BCGs, among the four candidates, are the best halo-center tracers. We model the lensing signals using a combination of four contributions: off-centered NFW host halo profile, sub-halo contribution, stellar contribution, and projected 2-halo term. We sample the posterior of 5 parameters i.e., halo mass, concentration, off-centering distance, sub halo mass, and fraction of subhalos via a MCMC package using the galaxy-galaxy lensing signals. After taking into account the sampling effects (e.g. Eddington bias), we found the best fit halo masses obtained from lensing signals are quite consistent with those obtained in the group catalog based on an abundance matching method, except in the lowest mass bin. Subject headings: (cosmology:) gravitational lensing, galaxies: clusters: general

astro-ph.CO↗

arXiv2017

Deformable Deep Convolutional Generative Adversarial Network in Microwave Based Hand Gesture Recognition System

Traditional vision-based hand gesture recognition systems is limited under dark circumstances. In this paper, we build a hand gesture recognition system based on microwave transceiver and deep learning algorithm. A Doppler radar sensor with dual receiving channels at 5.8GHz is used to acquire a big database of hand gestures signals. The received hand gesture signals are then processed with time-frequency analysis. Based on these big databases of hand gesture, we propose a new machine learning architecture called deformable deep convolutional generative adversarial network. Experimental results show the new architecture can upgrade the recognition rate by 10% and the deformable kernel can reduce the testing time cost by 30%.

stat.ML↗

arXiv2017

Doppler-Radar Based Hand Gesture Recognition System Using Convolutional Neural Networks

Hand gesture recognition has long been a hot topic in human computer interaction. Traditional camera-based hand gesture recognition systems cannot work properly under dark circumstances. In this paper, a Doppler Radar based hand gesture recognition system using convolutional neural networks is proposed. A cost-effective Doppler radar sensor with dual receiving channels at 5.8GHz is used to acquire a big database of four standard gestures. The received hand gesture signals are then processed with time-frequency analysis. Convolutional neural networks are used to classify different gestures. Experimental results verify the effectiveness of the system with an accuracy of 98%. Besides, related factors such as recognition distance and gesture scale are investigated.

cs.CV↗

arXiv2017

Investigating Inner Properties of Multimodal Representation and Semantic Compositionality with Brain-based Componential Semantics

Multimodal models have been proven to outperform text-based approaches on learning semantic representations. However, it still remains unclear what properties are encoded in multimodal representations, in what aspects do they outperform the single-modality representations, and what happened in the process of semantic compositionality in different input modalities. Considering that multimodal models are originally motivated by human concept representations, we assume that correlating multimodal representations with brain-based semantics would interpret their inner properties to answer the above questions. To that end, we propose simple interpretation methods based on brain-based componential semantics. First we investigate the inner properties of multimodal representations by correlating them with corresponding brain-based property vectors. Then we map the distributed vector space to the interpretable brain-based componential space to explore the inner properties of semantic compositionality. Ultimately, the present paper sheds light on the fundamental questions of natural language understanding, such as how to represent the meaning of words and how to combine word meanings into larger units.

cs.CL↗