arXiv subjects

Zhiqi Huang

Publications and source records attributed to Zhiqi Huang.

At least 73 records · Page 4Linked to original sources

arXiv2023

Improving Cross-lingual Information Retrieval on Low-Resource Languages via Optimal Transport Distillation

Benefiting from transformer-based pre-trained language models, neural ranking models have made significant progress. More recently, the advent of multilingual pre-trained language models provides great support for designing neural cross-lingual retrieval models. However, due to unbalanced pre-training data in different languages, multilingual language models have already shown a performance gap between high and low-resource languages in many downstream tasks. And cross-lingual retrieval models built on such pre-trained models can inherit language bias, leading to suboptimal result for low-resource languages. Moreover, unlike the English-to-English retrieval task, where large-scale training collections for document ranking such as MS MARCO are available, the lack of cross-lingual retrieval data for low-resource language makes it more challenging for training cross-lingual retrieval models. In this work, we propose OPTICAL: Optimal Transport distillation for low-resource Cross-lingual information retrieval. To transfer a model from high to low resource languages, OPTICAL forms the cross-lingual token alignment task as an optimal transport problem to learn from a well-trained monolingual retrieval model. By separating the cross-lingual knowledge from knowledge of query document matching, OPTICAL only needs bitext data for distillation training, which is more feasible for low-resource languages. Experimental results show that, with minimal training data, OPTICAL significantly outperforms strong baselines on low-resource languages, including neural machine translation.

cs.CL↗

arXiv2022

Cosmological Constraint Precision of the Photometric and Spectroscopic Multi-probe Surveys of China Space Station Telescope (CSST)

As one of Stage IV space-based telescopes, China Space Station Telescope (CSST) can perform photometric and spectroscopic surveys simultaneously to efficiently explore the Universe in extreme precision. In this work, we investigate several powerful CSST cosmological probes, including cosmic shear, galaxy-galaxy lensing, photometric and spectroscopic galaxy clustering, and number counts of galaxy clusters, and study the capability of these probes by forecasting the results of joint constraints on the cosmological parameters. By referring to real observational results, we generate mock data and estimate the measured errors based on CSST observational and instrumental designs. To study the systematical effects on the results, we also consider a number of systematics in CSST photometric and spectroscopic surveys, such as the intrinsic alignment, shear calibration uncertainties, photometric redshift uncertainties, galaxy bias, non-linear effects, instrumental effects, etc. The Fisher matrix method is used to derive the constraint results from individual or joint surveys on the cosmological and systematical parameters. We find that the joint constraints by including all these CSST cosmological probes can significantly improve the results from current observations by one order of magnitude at least, which gives $Ω_m$ and $σ_8$ $<$1% accuracy, and $w_0$ and $w_a$ $<$5% and 20% accuracies, respectively. This indicates that the CSST photometric and spectroscopic multi-probe surveys could provide powerful tools to explore the Universe and greatly improve the studies of relevant cosmological problems.

astro-ph.CO↗

arXiv2022

Constraints on compact dark matter from lensing of gravitational waves for the third-generation gravitational wave detector

Since the first gravitational wave (GW) event from binary black hole (BBH) was detected by LIGO-Virgo, GWs have become a useful probe on astrophysics and cosmology. If compact dark matter (DM) objects e.g. primordial black holes, contribute a significant fraction of dark matter at wide mass range, they will cause microlensing in the GW signals with long wavelengths that are distinct from the lensing effects of electromagnetic signals from astrophysical objects. In this paper, we apply the lensing effect of GW from BBH to derive constraints on the abundance of compact DM for the Cosmic Explorer, a third-generation ground-based GW detector. We firstly consider two channels of formation of BBH that contribute to low and high redshift GW sources, including the astrophysical origin BBH scenario, and the primordial origin BBH scenario. Secondly, comparing with the method of optical depth, we use the Bayesian analysis to derive constraints on the abundance of compact DM with different mass function of lens taken into consideration. For a null search with $1000$ detected GW events of BBH, we find that the abundance of compact DM could be constrained to $\lesssim0.1\%$ in the mass range $\geq500~M_{\odot}$ at $68\%$ confidence level. In addition, if a GW event lensed by a compact DM object with $M_{\rm l}\in[100~M_{\odot},300~M_{\odot}]$ is detected in $100$ detected GW events of BBH, we can derive that the estimation of the abundance of compact DM is from $2.3\%$ to $25.2\%$ in this mass range with the Bayesian analysis.

astro-ph.CO↗

arXiv2022

Tomographic Alcock-Paczynski Method with Redshift Errors

The tomographic Alcock-Paczynski (AP) method is a promising method that uses the redshift evolution of the anisotropic clustering in redshift space to calibrate cosmology. It extends the applicable range of AP method to substantially nonlinear scales, yielding very tight cosmological constraints. For future stage-IV slitless spectroscopic surveys, the non-negligible redshift errors might reduce the advantage of the tomographic AP method by suppressing the resolution of the nonlinear structure along the line of sight. The present work studies how redshift errors propagate to cosmological parameters in the tomographic AP analysis. We use a formula $σ_z = σ(1+z)^α $ to model the redshift errors, with $σ$ varying from 0.001 to 0.006 and $α$ varying from 0.5 to 1.5. The redshift errors produce a signal of anisotropic clustering that is similar to a strong finger-of-god effect, which smears out both the AP signal and the contamination caused by the redshift space distortions (RSD). For the target precision of the Chinese Space Station Telescope optical survey ($σ\lesssim 0.002$), the decrement of constraining power on the dark energy equation of state is mild ($\lesssim 50\%$), and the suppression of RSD contamination leads to a smaller bias-to-signal ratio. Our results indicate that the tomographic AP method will remain a useful and complementary tool for analyses of future slitless spectroscopic surveys.

astro-ph.CO↗

arXiv2022

The Galactic interstellar medium has a preferred handedness of magnetic misalignment

The Planck mission detected a positive correlation between the intensity ($T$) and $B$-mode polarization of the Galactic thermal dust emission. The $TB$ correlation is a parity-odd signal, whose statistical mean vanishes in models with mirror symmetry. Recent work has shown with strong evidence that local handedness of the misalignment between the dust filaments and the sky-projected magnetic field produces $TB$ signals. However, it remains unclear whether the observed global $TB$ signal is caused by statistical fluctuations of magnetic misalignment angles, or whether some parity-violating physics in the interstellar medium sets a preferred misalignment handedness. The present work aims to make a quantitative statement about how confidently the statistical-fluctuation interpretation is ruled out by filament-based simulations of polarized dust emission. We use the publicly available DUSTFILAMENTS code to simulate the dust emission from filaments whose magnetic misalignment angles are symmetrically randomized, and construct the probability density function of $ξ_{p}$, a weighted sum of $TB$ power spectrum. We find that Planck data has a $\gtrsim 10σ$ tension with the simulated $ξ_{p}$ distribution. Our results strongly support that the Galactic filament misalignment has a preferred handedness, whose physical origin is yet to be identified.

astro-ph.GA↗

arXiv2022

Revisiting the quasi-molecular mechanism of recombination

The quasi-molecular mechanism of recombination, recently suggested by Kereselidze et al., is a non-standard process where an electron and two neighboring protons in the early universe directly form an ionized hydrogen molecule in a highly excited state, which then descends to lower levels or dissociates. It has been suggested that the increased binding energy due to the participation of a second proton may lead to an earlier cosmic recombination that alleviates the Hubble tension. Revisiting the quasi-molecular channel of recombination in more details, we find that the original work significantly overestimated the probability of finding a pair of adjacent protons in the relevant epoch ($z\sim $ a few thousand). Our new estimation suggests that the quasi-molecular mechanism of recombination cannot be the primary cause of the Hubble tension.

astro-ph.CO↗

arXiv2022

Thawing k-essence dark energy in the PAge space

A broad class of dark energy models can be written in the form of k-essence, whose Lagrangian density is a two-variable function of a scalar field $ϕ$ and its kinetic energy $X\equiv \frac{1}{2}\partial^μϕ\partial_μϕ$. In the thawing scenario, the scalar field becomes dynamic only when the Hubble friction drops below its mass scale in the late universe. Thawing k-essence dark energy models can be randomly sampled by generating the Taylor expansion coefficients of its Lagrangian density from random matrices \cite{thaws}. Ref. \cite{thaws} points out that the non-uniform distribution of effective equation of state parameters $(w_0, w_a)$ of thawing k-essence model can be used to improve the statistics of model selection. The present work studies the statistics of thawing k-essence in a more general framework that is Parameterized by the Age of the universe (PAge) \cite{PAge}. For fixed matter fraction $Ω_m$, the random thawing k-essence models cluster in a narrow band in the PAge parameter space, providing a strong theoretical prior. We simulate cosmic shear power spectrum data for the Chinese Space Station Telescope optical survey, and compare the fisher forecast with and without the theoretical prior of thawing k-essence. For an optimal tomography binning scheme, the theoretical prior improves the figure of merit in PAge space by a factor of $3.3$.

astro-ph.CO↗

arXiv2022

Cosmological constraints from the density gradient weighted correlation function

The mark weighted correlation function (MCF) $W(s,μ)$ is a computationally efficient statistical measure which can probe clustering information beyond that of the conventional 2-point statistics. In this work, we extend the traditional mark weighted statistics by using powers of the density field gradient $|\nabla ρ/ρ|^α$ as the weight, and use the angular dependence of the scale-averaged MCFs to constrain cosmological parameters. The analysis shows that the gradient based weighting scheme is statistically more powerful than the density based weighting scheme, while combining the two schemes together is more powerful than separately using either of them. Utilising the density weighted or the gradient weighted MCFs with $α=0.5,\ 1$, we can strengthen the constraint on $Ω_m$ by factors of 2 or 4, respectively, compared with the standard 2-point correlation function, while simultaneously using the MCFs of the two weighting schemes together can be $1.25$ times more statistically powerful than using the gradient weighting scheme alone. The mark weighted statistics may play an important role in cosmological analysis of future large-scale surveys. Many issues, including the possibility of using other types of weights, the influence of the bias on this statistics, as well as the usage of MCFs in the tomographic Alcock-Paczynski method, are worth further investigations.

astro-ph.CO↗

arXiv2022

Constraints on the abundance of supermassive primordial black holes from lensing of compact radio sources

The possibility that primordial black holes (PBHs) form a part of dark matter has been considered over a wide mass range from the Planck mass ($10^{-5}~\rm g$) to the level of the supermassive black hole in the center of the galaxy. Primordial origin might be one of the most important formation channel of supermassive black holes. We use the non-detection of lensing effect of very long baseline interferometer observations of compact radio sources with extremely high angular resolution as a promising probe to constrain the abundance of intergalactic PBHs in the mass range $\sim10^4$-$10^9~M_{\odot}$. For a sample of well-measured 543 flat-spectrum compact radio sources, no milli-lensed images are found with angular separations between $1.5$ milli-arcseconds and $50$ milli-arcseconds. From this null search result, we derive that the fraction of dark matter made up of supermassive PBHs in the mass range $\sim10^6$-$10^8~M_{\odot}$ is $\lesssim1.48\%$ at $95\%$ confidence level. This constraints would be significantly improved due to the rapid increase of the number of measured compact radio sources. For instance, on the basis of none confirmed milli-lensing candidate in the latest $\sim14000$ sources, we derive the abundance of supermassive PBHs and obtain that it is $\lesssim0.06\%$ at $95\%$ confidence level.

astro-ph.CO↗

arXiv2022

Search for lensing signatures from the latest fast radio burst observations and constraints on the abundance of primordial black holes

The possibility that primordial black holes (PBHs) form a part of dark matter has been considered for a long time but poorly constrained over a wide mass range. Fast radio bursts (FRBs) are bright radio transients with millisecond duration. Lensing effect of them has been proposed as one of the cleanest probes for constraining the presence of PBHs in the stellar mass window. In this paper, we first apply the normalised cross-correlation algorithm to search and identify candidates of lensed FRBs in the latest public FRB observations, i.e. $593$ FRBs which mainly consist of the first Canadian Hydrogen Intensity Mapping Experiment FRB catalog, and then derive constraints on the abundance of PBHs from the null search result of lensing signature. For a monochromatic mass distribution, the fraction of dark matter made up of PBHs could be constrained to $\leq87\%$ for $\geq500~M_{\odot}$ at 95\% confidence level by assuming flux ratio thresholds dependent signal-to-noise ratio for each FRB and that apparently one-off events are intrinsic single bursts. This result would be improved by a three times factor when a conventional constant flux ratio threshold is considered. Moreover, we derive constraints on PBHs with a log-normal mass function naturally predicted by some popular inflation models and often investigated with gravitational wave detections. We find that, in this mass distribution scenario, the constraint from currently public FRB observations is relatively weaker than the one from gravitational wave detections. It is foreseen that upcoming complementary multi-messenger observations will yield considerable constraints on the possibilities of PBHs in this intriguing mass window.

astro-ph.CO↗

arXiv2022

Constraints on the abundance of primordial black holes with different mass distributions from lensing of fast radio bursts

Primordial black holes (PBHs) has been considered to form a part of dark matter for a long time but the possibility has been poorly constrained over a wide mass range, including the stellar mass range ($1-100~M_{\odot}$). However, due to the discovery of merger events of black hole binaries by LIGO-Virgo gravitational wave observatories, the interest for PBHs in the stellar mass window has been aroused again. Fast radio bursts (FRBs) are bright radio transients with millisecond duration and very high all-sky occurrence rate. Lensing effect of these bursts has been proposed as one of the optimal probes for constraining the abundance of PBHs in the stellar mass range. In this paper, we first investigate constraints on the abundance of PBHs from the latest $593$ FRB observations for both the monochromatic mass distribution and three other popular extended mass distributions related to different formation mechanisms of PBHs. It is found that constraints from currently public FRB observations are relatively weaker than those from existing gravitational wave detections. Furthermore, we forecast constraining power of future FRB observations on the abundance of PBHs with different mass distributions of PBHs and different redshift distributions of FRBs taken into account. Finally, We find that constraints of parameter space on extended mass distributions from $\sim10^5$ FRBs with $\overline{Δt}\leq1 ~\rm ms$ would be comparable with what can be constrained from gravitational wave events. It is foreseen that upcoming complementary multi-messenger observations will yield considerable constraints on the possibilities of PBHs in this intriguing mass window.

astro-ph.CO↗

arXiv2022

The $S_8$ Tension in Light of Updated Redshift-Space Distortion Data and PAge Approximation

One of the most prominent challenges to the standard Lambda cold dark matter ($Λ$CDM) cosmology is the tension between the structure growth parameter $S_8$ constrained by the cosmic microwave background (CMB) data and the smaller one suggested by the cosmic shear data. Recent studies show that, for $Λ$CDM cosmology, redshift-space distortion (RSD) data also prefers a smaller $S_8$ that is $\sim 2$-$3σ$ lower than the CMB value, but the result is sensitive to the cosmological model. In the present work we update the RSD constraint on $S_8$ with the most up-to-date RSD data set where the correlation between data points are properly taken into account. To reduce the model dependence, we add in our Monte Carlo Markov Chain calculation the most up-to-date data sets of Type Ia supernovae (SN) and baryon acoustic oscillations (BAO), whose correlation with RSD is also taken into account, to constrain the background geometry. For $Λ$CDM cosmology we find $S_8= 0.812 \pm 0.026$, which is $\sim 2σ$ larger than previous studies, and hence is consistent with the CMB constraint. By replacing $Λ$CDM with the Parameterization based on cosmic Age (PAge), an almost model-independent description of the late universe, we find that the RSD + SN + BAO constraint on $S_8$ is insensitive to the cosmological model.

astro-ph.CO↗

arXiv2021

Statistics of Thawing K-essence Dark Energy Models

K-essence is a minimally-coupled scalar field whose Lagrangian density $\mathcal{L}$ is a function of the field value $ϕ$ and the kinetic energy $X=\frac{1}{2}\partial_μϕ\partial^μϕ$. In the thawing scenario, the scalar field is frozen by the large Hubble friction in the early universe, and therefore initial conditions are specified. We construct thawing k-essence models by generating Taylor expansion coefficients of $\mathcal{L}(ϕ, X)$ from random matrices. From the ensemble of randomly generated thawing k-essence models, we select dark energy candidates by assuming negative pressure and non-growth of sub-horizon inhomogeneities. For each candidate model the dark energy equation of state function is fit to the Chevallier-Polarski-Linder parameterization $w(a) \approx w_0+w_a(1-a)$, where $a$ is the scale factor. The thawing k-essence dark models distribute very non-uniformly in the $(w_0, w_a)$ space. About 90\% models cluster in a narrow band in the proximity of a slow-roll line $w_a\approx -1.42 \left(\frac{Ω_m}{0.3}\right)^{0.64}(1+w_0)$, where $Ω_m$ is the present matter density fraction. This work is a proof of concept that for a certain class of models very non-uniform theoretical prior on $(w_0, w_a)$ can be obtained to improve the statistics of model selection.

astro-ph.CO↗

arXiv2021

Towards Joint Intent Detection and Slot Filling via Higher-order Attention

Intent detection (ID) and Slot filling (SF) are two major tasks in spoken language understanding (SLU). Recently, attention mechanism has been shown to be effective in jointly optimizing these two tasks in an interactive manner. However, latest attention-based works concentrated only on the first-order attention design, while ignoring the exploration of higher-order attention mechanisms. In this paper, we propose a BiLinear attention block, which leverages bilinear pooling to simultaneously exploit both the contextual and channel-wise bilinear attention distributions to capture the second-order interactions between the input intent or slot features. Higher and even infinity order interactions are built by stacking numerous blocks and assigning Exponential Linear Unit (ELU) to blocks. Before the decoding stage, we introduce the Dynamic Feature Fusion Layer to implicitly fuse intent and slot information in a more effective way. Technically, instead of simply concatenating intent and slot features, we first compute two correlation matrices to weight on two features. Furthermore, we present Higher-order Attention Network for the SLU tasks. Experiments on two benchmark datasets show that our approach yields improvements compared with the state-of-the-art approach. We also provide discussion to demonstrate the effectiveness of the proposed approach.

cs.CL↗

arXiv2021

Mixed Attention Transformer for Leveraging Word-Level Knowledge to Neural Cross-Lingual Information Retrieval

Pretrained contextualized representations offer great success for many downstream tasks, including document ranking. The multilingual versions of such pretrained representations provide a possibility of jointly learning many languages with the same model. Although it is expected to gain big with such joint training, in the case of cross lingual information retrieval (CLIR), the models under a multilingual setting are not achieving the same level of performance as those under a monolingual setting. We hypothesize that the performance drop is due to the translation gap between query and documents. In the monolingual retrieval task, because of the same lexical inputs, it is easier for model to identify the query terms that occurred in documents. However, in the multilingual pretrained models that the words in different languages are projected into the same hyperspace, the model tends to translate query terms into related terms, i.e., terms that appear in a similar context, in addition to or sometimes rather than synonyms in the target language. This property is creating difficulties for the model to connect terms that cooccur in both query and document. To address this issue, we propose a novel Mixed Attention Transformer (MAT) that incorporates external word level knowledge, such as a dictionary or translation table. We design a sandwich like architecture to embed MAT into the recent transformer based deep neural models. By encoding the translation knowledge into an attention matrix, the model with MAT is able to focus on the mutually translated words in the input sequence. Experimental results demonstrate the effectiveness of the external knowledge and the significant improvement of MAT embedded neural reranking model on CLIR task.

cs.IR↗

arXiv2021

HAN: Higher-order Attention Network for Spoken Language Understanding

Spoken Language Understanding (SLU), including intent detection and slot filling, is a core component in human-computer interaction. The natural attributes of the relationship among the two subtasks make higher requirements on fine-grained feature interaction, i.e., the token-level intent features and slot features. Previous works mainly focus on jointly modeling the relationship between the two subtasks with attention-based models, while ignoring the exploration of attention order. In this paper, we propose to replace the conventional attention with our proposed Bilinear attention block and show that the introduced Higher-order Attention Network (HAN) brings improvement for the SLU task. Importantly, we conduct wide analysis to explore the effectiveness brought from the higher-order attention.

cs.CL↗

arXiv2021

A More Accurate Parameterization based on cosmic Age (MAPAge)

Recently, several statistically significant tensions between different cosmological datasets have raised doubts about the standard Lambda cold dark matter ($Λ$CDM) model. A recent letter~\citet{Huang:2020mub} suggests to use "Parameterization based on cosmic Age" (PAge) to approximate a broad class of beyond-$Λ$CDM models, with a typical accuracy $\sim 1\%$ in angular diameter distances at $z\lesssim 10$. In this work, we extend PAge to a More Accurate Parameterization based on cosmic Age (MAPAge) by adding a new degree of freedom $η_2$. The parameter $η_2$ describes the difference between physically motivated models and their phenomenological PAge approximations. The accuracy of MAPAge, typically of order $10^{-3}$ in angular diameter distances at $z\lesssim 10$, is significantly better than PAge. We compare PAge and MAPAge with current observational data and forecast data. The conjecture in~\citet{Huang:2020mub}, that PAge approximation is sufficiently good for current observations, is quantitatively confirmed in this work. We also show that the extension from PAge to MAPAge is important for future observations, which typically requires sub-percent accuracy in theoretical predictions.

astro-ph.CO↗

arXiv2021

Audio-Oriented Multimodal Machine Comprehension: Task, Dataset and Model

While Machine Comprehension (MC) has attracted extensive research interests in recent years, existing approaches mainly belong to the category of Machine Reading Comprehension task which mines textual inputs (paragraphs and questions) to predict the answers (choices or text spans). However, there are a lot of MC tasks that accept audio input in addition to the textual input, e.g. English listening comprehension test. In this paper, we target the problem of Audio-Oriented Multimodal Machine Comprehension, and its goal is to answer questions based on the given audio and textual information. To solve this problem, we propose a Dynamic Inter- and Intra-modality Attention (DIIA) model to effectively fuse the two modalities (audio and textual). DIIA can work as an independent component and thus be easily integrated into existing MC models. Moreover, we further develop a Multimodal Knowledge Distillation (MKD) module to enable our multimodal MC model to accurately predict the answers based only on either the text or the audio. As a result, the proposed approach can handle various tasks including: Audio-Oriented Multimodal Machine Comprehension, Machine Reading Comprehension and Machine Listening Comprehension, in a single model, making fair comparisons possible between our model and the existing unimodal MC models. Experimental results and analysis prove the effectiveness of the proposed approaches. First, the proposed DIIA boosts the baseline models by up to 21.08% in terms of accuracy; Second, under the unimodal scenarios, the MKD module allows our multimodal MC model to significantly outperform the unimodal models by up to 18.87%, which are trained and tested with only audio or textual data.

cs.CL↗