arXiv subjects

Jiajun Zhang

Publications and source records attributed to Jiajun Zhang.

At least 199 records · Page 11Linked to original sources

arXiv2021

Exploiting Curriculum Learning in Unsupervised Neural Machine Translation

Back-translation (BT) has become one of the de facto components in unsupervised neural machine translation (UNMT), and it explicitly makes UNMT have translation ability. However, all the pseudo bi-texts generated by BT are treated equally as clean data during optimization without considering the quality diversity, leading to slow convergence and limited translation performance. To address this problem, we propose a curriculum learning method to gradually utilize pseudo bi-texts based on their quality from multiple granularities. Specifically, we first apply cross-lingual word embedding to calculate the potential translation difficulty (quality) for the monolingual sentences. Then, the sentences are fed into UNMT from easy to hard batch by batch. Furthermore, considering the quality of sentences/tokens in a particular batch are also diverse, we further adopt the model itself to calculate the fine-grained quality scores, which are served as learning factors to balance the contributions of different parts when computing loss and encourage the UNMT model to focus on pseudo data with higher quality. Experimental results on WMT 14 En-Fr, WMT 16 En-De, WMT 16 En-Ro, and LDC En-Zh translation tasks demonstrate that the proposed method achieves consistent improvements with faster convergence speed.

cs.CL↗

arXiv2021

Numerical convergence of pre-initial conditions on dark matter halo properties

Generating pre-initial conditions (or particle loads) is the very first step to set up a cosmological N-body simulation. In this work, we revisit the numerical convergence of pre-initial conditions on dark matter halo properties using a set of simulations which only differs in initial particle loads, i.e. grid, glass, and the newly introduced capacity constrained Voronoi tessellation (CCVT). We find that the median halo properties agree fairly well (i.e. within a convergence level of a few per cent) among simulations running from different initial loads. We also notice that for some individual haloes cross-matched among different simulations, the relative difference of their properties sometimes can be several tens of per cent. By looking at the evolution history of these poorly converged haloes, we find that they are usually merging haloes or haloes have experienced recent merger events, and their merging processes in different simulations are out-of-sync, making the convergence of halo properties become poor temporarily. We show that, comparing to the simulation starting with an anisotropic grid load, the simulation with an isotropic CCVT load converges slightly better to the simulation with a glass load, which is also isotropic. Among simulations with different pre-initial conditions, haloes in higher density environments tend to have their properties converged slightly better. Our results confirm that CCVT loads behave as well as the widely used grid and glass loads at small scales, and for the first time we quantify the convergence of two independent isotropic particle loads (i.e. glass and CCVT) on halo properties.

astro-ph.CO↗

arXiv2021

CSDS: A Fine-Grained Chinese Dataset for Customer Service Dialogue Summarization

Dialogue summarization has drawn much attention recently. Especially in the customer service domain, agents could use dialogue summaries to help boost their works by quickly knowing customer's issues and service progress. These applications require summaries to contain the perspective of a single speaker and have a clear topic flow structure, while neither are available in existing datasets. Therefore, in this paper, we introduce a novel Chinese dataset for Customer Service Dialogue Summarization (CSDS). CSDS improves the abstractive summaries in two aspects: (1) In addition to the overall summary for the whole dialogue, role-oriented summaries are also provided to acquire different speakers' viewpoints. (2) All the summaries sum up each topic separately, thus containing the topic-level structure of the dialogue. We define tasks in CSDS as generating the overall summary and different role-oriented summaries for a given dialogue. Next, we compare various summarization methods on CSDS, and experiment results show that existing methods are prone to generate redundant and incoherent summaries. Besides, the performance becomes much worse when analyzing the performance on role-oriented summaries and topic structures. We hope that this study could benchmark Chinese dialogue summarization and benefit further studies.

cs.CL↗

arXiv2021

Augmenting Slot Values and Contexts for Spoken Language Understanding with Pretrained Models

Spoken Language Understanding (SLU) is one essential step in building a dialogue system. Due to the expensive cost of obtaining the labeled data, SLU suffers from the data scarcity problem. Therefore, in this paper, we focus on data augmentation for slot filling task in SLU. To achieve that, we aim at generating more diverse data based on existing data. Specifically, we try to exploit the latent language knowledge from pretrained language models by finetuning them. We propose two strategies for finetuning process: value-based and context-based augmentation. Experimental results on two public SLU datasets have shown that compared with existing data augmentation methods, our proposed method can generate more diverse sentences and significantly improve the performance on SLU.

cs.CL↗

arXiv2021

OPT: Omni-Perception Pre-Trainer for Cross-Modal Understanding and Generation

In this paper, we propose an Omni-perception Pre-Trainer (OPT) for cross-modal understanding and generation, by jointly modeling visual, text and audio resources. OPT is constructed in an encoder-decoder framework, including three single-modal encoders to generate token-based embeddings for each modality, a cross-modal encoder to encode the correlations among the three modalities, and two cross-modal decoders to generate text and image respectively. For the OPT's pre-training, we design a multi-task pretext learning scheme to model multi-modal resources from three different data granularities, \ie, token-, modality-, and sample-level modeling, through which OPT learns to align and translate among different modalities. The pre-training task is carried out on a large amount of image-text-audio triplets from Open Images. Experimental results show that OPT can learn strong image-text-audio multi-modal representations and achieve promising results on a variety of cross-modal understanding and generation tasks.

cs.CV↗

arXiv2021

Gaze Estimation with an Ensemble of Four Architectures

This paper presents a method for gaze estimation according to face images. We train several gaze estimators adopting four different network architectures, including an architecture designed for gaze estimation (i.e.,iTracker-MHSA) and three originally designed for general computer vision tasks(i.e., BoTNet, HRNet, ResNeSt). Then, we select the best six estimators and ensemble their predictions through a linear combination. The method ranks the first on the leader-board of ETH-XGaze Competition, achieving an average angular error of $3.11^{\circ}$ on the ETH-XGaze test set.

cs.CV↗

arXiv2021

The BINGO Project VI: HI Halo Occupation Distribution and Mock Building

BINGO (Baryon Acoustic Oscillations from Integrated Neutral Gas Observations.) is a radio telescope designed to survey from 980 MHz to 1260 MHz, observe the neutral Hydrogen (HI) 21-cm line and detect BAO (Baryon Acoustic Oscillation) signal with Intensity Mapping technique. Here we present our method to generate mock maps of the 21-cm Intensity Mapping signal covering the BINGO frequency range and related test results. (Abridged)

astro-ph.CO↗

arXiv2021

Baryon Acoustic Oscillations from Integrated Neutral Gas Observations: an instrument to observe the 21cm hydrogen line in the redshift range 0.13 $<$ z $<$ 0.45 -- status update

BINGO (BAO from Integrated Neutral Gas Observations) is a unique radio telescope designed to map the intensity of neutral hydrogen distribution at cosmological distances, making the first detection of Baryon Acoustic Oscillations (BAO) in the frequency band 980 MHz - 1260 MHz, corresponding to a redshift range $0.127 < z < 0.449$. BAO is one of the most powerful probes of cosmological parameters and BINGO was designed to detect the BAO signal to a level that makes it possible to put new constraints on the equation of state of dark energy. The telescope will be built in Paraíba, Brazil and consists of two $\thicksim$ 40m mirrors, a feedhorn array of 28 horns, and no moving parts, working as a drift-scan instrument. It will cover a $15^{\circ}$ declination strip centered at $\sim δ=-15^{\circ}$, mapping $\sim 5400$ square degrees in the sky. The BINGO consortium is led by University of São Paulo with co-leadership at National Institute for Space Research and Campina Grande Federal University (Brazil). Telescope subsystems have already been fabricated and tested, and the dish and structure fabrication are expected to start in late 2020, as well as the road and terrain preparation.

astro-ph.CO↗

arXiv2021

Bilingual Mutual Information Based Adaptive Training for Neural Machine Translation

Recently, token-level adaptive training has achieved promising improvement in machine translation, where the cross-entropy loss function is adjusted by assigning different training weights to different tokens, in order to alleviate the token imbalance problem. However, previous approaches only use static word frequency information in the target language without considering the source language, which is insufficient for bilingual tasks like machine translation. In this paper, we propose a novel bilingual mutual information (BMI) based adaptive objective, which measures the learning difficulty for each target token from the perspective of bilingualism, and assigns an adaptive weight accordingly to improve token-level adaptive training. This method assigns larger training weights to tokens with higher BMI, so that easy tokens are updated with coarse granularity while difficult tokens are updated with fine granularity. Experimental results on WMT14 English-to-German and WMT19 Chinese-to-English demonstrate the superiority of our approach compared with the Transformer baseline and previous token-level adaptive training approaches. Further analyses confirm that our method can improve the lexical diversity.

cs.CL↗

arXiv2021

Emergent gravity fails to explain color-dependent galaxy-galaxy lensing signals from SDSS Dr7

We test the Emergent Gravity(EG) theory using the galaxy-galaxy lensing technique based on SDSS DR7 data. In the EG scenario, we do not expect color dependence of the galaxy sample in the 'apparent dark matter' predicted by EG, which is exerted only by the baryonic mass. If the baryonic mass is similar, then the predicted lensing profiles from the baryonic mass should be similar according to the EG, regardless of the color of the galaxy sample. We use the stellar mass of the galaxy as a proxy of its baryonic mass. We divide our galaxy sample into 5 stellar mass bins, and further classify them as red and blue subsamples in each stellar mass bin. If we set halo mass and concentration as free parameters, $Λ$CDM is favored by our data in terms of the reduced $χ^2$ while EG fails to explain the color dependence of ESDs from the galaxy-galaxy lensing measurement.

astro-ph.GA↗

arXiv2021

Towards a Bias-Free Selection Function in Shear Measurement

Sample selection is a necessary preparation for weak lensing measurement. It is well-known that selection itself may introduce bias in the measured shear signal. Using image simulation and the Fourier_Quad shear measurement pipeline, we quantify the selection bias in various commonly used selection function (signal-to-noise-ratio, magnitude, etc.). We proposed a new selection function defined in the power spectrum of the galaxy image. This new selection function has low selection bias, and it is particularly convenient for shear measurement pipelines based on Fourier transformation.

astro-ph.CO↗

arXiv2020

Nearly 30,000 late-type main-sequence stars with stellar age from LAMOST DR5

We construct a sample of nearly 30,000 main-sequence stars with 4500K $<T\rm_{eff}<$ 5000K and stellar ages estimated by the chromospheric activity$-$age relation. This sample is used to determine the age distribution in the $R-Z$ plane of the Galaxy, where $R$ is the projected Galactocentric distance in the disk midplane and $Z$ is the height above the disk midplane. As $|Z|$ increases, the percentage of old stars becomes larger. It is known that scale-height of Galactic disk increases as $R$ increases, which is called flare. A mild flare from $R$ $\sim$ 8.0 to 9.0 kpc in stellar age distribution is found. We also find that the velocity dispersion increases with age as confirmed by previous studies. Finally we present spiral-shaped structures in $Z-\upsilon_{Z}$ phase space in three stellar age bins. The spiral is clearly seen in the age bin of [0, 1] Gyr, which suggests that a vertical perturbation to the disk probably took place within the last $\sim$ 1.0 Gyr.

astro-ph.GA↗

arXiv2020

Deep Template Matching for Pedestrian Attribute Recognition with the Auxiliary Supervision of Attribute-wise Keypoints

Pedestrian Attribute Recognition (PAR) has aroused extensive attention due to its important role in video surveillance scenarios. In most cases, the existence of a particular attribute is strongly related to a partial region. Recent works design complicated modules, e.g., attention mechanism and proposal of body parts to localize the attribute corresponding region. These works further prove that localization of attribute specific regions precisely will help in improving performance. However, these part-information-based methods are still not accurate as well as increasing model complexity which makes it hard to deploy on realistic applications. In this paper, we propose a Deep Template Matching based method to capture body parts features with less computation. Further, we also proposed an auxiliary supervision method that use human pose keypoints to guide the learning toward discriminative local cues. Extensive experiments show that the proposed method outperforms and has lower computational complexity, compared with the state-of-the-art approaches on large-scale pedestrian attribute datasets, including PETA, PA-100K, RAP, and RAPv2 zs.

cs.CV↗

arXiv2020

Estimating Atmospheric Parameters of DA White Dwarf Stars with Deep Learning

The determination of atmospheric parameters of white dwarf stars (WDs) is crucial for researches on them. Traditional methodology is to fit the model spectra to observed absorption lines and report the parameters with the lowest $χ^2$ error, which strongly relies on theoretical models that are not always publicly accessible. In this work, we construct a deep learning network to model-independently estimate Teff and log g of DA stars (DAs), corresponding to WDs with hydrogen dominated atmospheres. The network is directly trained and tested on the normalized flux pixels of full optical wavelength range of DAs spectroscopically confirmed in the Sloan Digital Sky Survey (SDSS). Experiments in test part yield that the root mean square error (RMSE) for Teff and log g approaches to 900 K and 0.1 dex, respectively. This technique is applicable for those DAs with Teff from 5000 K to 40000 K and log g from 7.0 dex to 9.0 dex. Furthermore, the applicability of this method is verified for the spectra with degraded resolution $\sim 200$. So it is also practical for the analysis of DAs that will be detected by the Chinese Space Station Telescope (CSST).

astro-ph.IM↗

arXiv2020

Bridging the Modality Gap for Speech-to-Text Translation

End-to-end speech translation aims to translate speech in one language into text in another language via an end-to-end way. Most existing methods employ an encoder-decoder structure with a single encoder to learn acoustic representation and semantic information simultaneously, which ignores the speech-and-text modality differences and makes the encoder overloaded, leading to great difficulty in learning such a model. To address these issues, we propose a Speech-to-Text Adaptation for Speech Translation (STAST) model which aims to improve the end-to-end model performance by bridging the modality gap between speech and text. Specifically, we decouple the speech translation encoder into three parts and introduce a shrink mechanism to match the length of speech representation with that of the corresponding text transcription. To obtain better semantic representation, we completely integrate a text-based translation model into the STAST so that two tasks can be trained in the same latent space. Furthermore, we introduce a cross-modal adaptation method to close the distance between speech and text representation. Experimental results on English-French and English-German speech translation corpora have shown that our model significantly outperforms strong baselines, and achieves the new state-of-the-art performance.

cs.CL↗

arXiv2020

Dynamic Context Selection for Document-level Neural Machine Translation via Reinforcement Learning

Document-level neural machine translation has yielded attractive improvements. However, majority of existing methods roughly use all context sentences in a fixed scope. They neglect the fact that different source sentences need different sizes of context. To address this problem, we propose an effective approach to select dynamic context so that the document-level translation model can utilize the more useful selected context sentences to produce better translations. Specifically, we introduce a selection module that is independent of the translation module to score each candidate context sentence. Then, we propose two strategies to explicitly select a variable number of context sentences and feed them into the translation module. We train the two modules end-to-end via reinforcement learning. A novel reward is proposed to encourage the selection and utilization of dynamic context sentences. Experiments demonstrate that our approach can select adaptive context sentences for different source sentences, and significantly improves the performance of document-level translation methods.

cs.CL↗

arXiv2020

Testing interacting dark matter and dark energy model with cosmological data

We investigate the model of dark matter-dark energy (DM-DE) interaction with coupling strength proportional to the multiplication of dark sector densities with different power indices $Q = γρ_{\rm c}^α ρ_{\rm d}^β$. We first investigate the modification of the cosmic expansion history, and then further develop the formalism to take into account the cosmological perturbations and dark matter temperature evolution. We then use the latest observational cosmology data, including cosmic microwave background (CMB) data, baryon acoustic oscillations (BAO) data, redshift-space distortion (RSD) data and Type Ia supernovae (SNe) data to constrain the model parameters. We find in the phantom region, a positive $α$ is preferred by the data above $2\, σ$ statistic significance. If we choose the power indices to be integers or half-integers for {\it plausible} physics of particle interaction, the allowed values within $1\, σ$ confidence regions are $α= 0.5$ and $β= 0, 0.5, 1$. The inclusion of BAO and RSD data from large-scale structure and SNe data improves the constraints significantly. Our model predicts lower values of $f(z) σ_8(z)$ at $z<1$ comparing to $Λ$CDM model, which alleviates the tension of $Λ$CDM with various RSD data from optical galaxy surveys. Overall, the DM-DE interaction model is consistent with the current observational data, especially providing a better fit to the RSD data.

astro-ph.CO↗

arXiv2020

A study on Cubic Galileon Gravity Using N-body Simulations

We use N-body simulation to study the structure formation in the Cubic Galileon Gravity model where along with the usual kinetic and potential term we also have a higher derivative self-interaction term. We find that the large scale structure provides a unique constraining power for this model. The matter power spectrum, halo mass function, galaxy-galaxy weak lensing signal, marked density power spectrum as well as count in cell are measured. The simulations show that there are less massive halos in the Cubic Galileon Gravity model than corresponding $Λ$CDM model and the marked density power spectrum in these two models are different by more than $10\%$. Furthermore, the Cubic Galileon model shows significant differences in voids compared to $Λ$CDM. The number of low density cells is far higher in the Cubic Galileon model than that in the $Λ$CDM model. Therefore, it would be interesting to put constraints on this model using future large scale structure observations, especially in void regions.

astro-ph.CO↗