arXiv subjects

Teng Hu

Publications and source records attributed to Teng Hu.

58 records · Page 4Linked to original sources

arXiv2023

DocPrompt: Large-scale continue pretrain for zero-shot and few-shot document question answering

In this paper, we propose Docprompt for document question answering tasks with powerful zero-shot and few-shot performance. We proposed a novel weakly supervised data generation method, a novel multl-stage training method and a novel understanding model \& generation model ensemble method. We achieved state-of-the-art performance on 4 document question answering tasks. This method greatly improves the delivery efficiency and model performance of document question answering customer projects, reducing annotation costs and labor costs. Our demo can be found at https://huggingface.co/spaces/PaddlePaddle/ERNIE-Layout.

cs.CL↗

arXiv2023

The Impact of the WHIM on the IGM Thermal State Determined from the Low-$z$ Lyman-$α$ Forest

At $z \lesssim 1$, shock heating caused by large-scale velocity flows and possibly violent feedback from galaxy formation, converts a significant fraction of the cool gas ($T\sim 10^4$ K) in the intergalactic medium (IGM) into warm-hot phase (WHIM) with $T >10^5$K, resulting in a significant deviation from the previously tight power-law IGM temperature-density relationship, $T=T_0 (ρ/ {\barρ})^{γ-1}$. This study explores the impact of the WHIM on measurements of the low-$z$ IGM thermal state, $[T_0,γ]$, based on the $b$-$N_{H I}$ distribution of the Lyman-$α$ forest. Exploiting a machine learning-enabled simulation-based inference method trained on Nyx hydrodynamical simulations, we demonstrate that [$T_0$, $γ$] can still be reliably measured from the $b$-$N_{H I}$ distribution at $z=0.1$, notwithstanding the substantial WHIM in the IGM. To investigate the effects of different feedback, we apply this inference methodology to mock spectra derived from the IllustrisTNG and Illustris simulations at $z=0.1$. The results suggest that the underlying $[T_0,γ]$ of both simulations can be recovered with biases as low as $|Δ\log(T_0/\text{K})| \lesssim 0.05$ dex, $|Δγ| \lesssim 0.1$, smaller than the precision of a typical measurement. Given the large differences in the volume-weighted WHIM fractions between the three simulations (Illustris 38\%, IllustrisTNG 10\%, Nyx 4\%) we conclude that the $b$-$N_{H I}$ distribution is not sensitive to the WHIM under realistic conditions. Finally, we investigate the physical properties of the detectable Lyman-$α$ absorbers, and discover that although their $T$ and $Δ$ distributions remain mostly unaffected by feedback, they are correlated with the photoionization rate used in the simulation.

astro-ph.CO↗

arXiv2022

ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding

Recent years have witnessed the rise and success of pre-training techniques in visually-rich document understanding. However, most existing methods lack the systematic mining and utilization of layout-centered knowledge, leading to sub-optimal performances. In this paper, we propose ERNIE-Layout, a novel document pre-training solution with layout knowledge enhancement in the whole workflow, to learn better representations that combine the features from text, layout, and image. Specifically, we first rearrange input sequences in the serialization stage, and then present a correlative pre-training task, reading order prediction, to learn the proper reading order of documents. To improve the layout awareness of the model, we integrate a spatial-aware disentangled attention into the multi-modal transformer and a replaced regions prediction task into the pre-training phase. Experimental results show that ERNIE-Layout achieves superior performance on various downstream tasks, setting new state-of-the-art on key information extraction, document image classification, and document question answering datasets. The code and models are publicly available at http://github.com/PaddlePaddle/PaddleNLP/tree/develop/model_zoo/ernie-layout.

cs.CL↗

arXiv2022

Measuring the thermal and ionization state of the low-$z$ IGM using likelihood free inference

We present a new approach to measure the power-law temperature density relationship $T=T_0 (ρ/ \barρ)^{γ-1}$ and the UV background photoionization rate $Γ_{\rm HI}$ of the IGM based on the Voigt profile decomposition of the Ly$α$ forest into a set of discrete absorption lines with Doppler parameter $b$ and the neutral hydrogen column density $N_{\rm HI}$. Previous work demonstrated that the shape of the $b$-$N_{\rm HI}$ distribution is sensitive to the IGM thermal parameters $T_0$ and $γ$, whereas our new inference algorithm also takes into account the normalization of the distribution, i.e. the line-density d$N$/d$z$, and we demonstrate that precise constraints can also be obtained on $Γ_{\rm HI}$. We use density-estimation likelihood-free inference (DELFI) to emulate the dependence of the $b$-$N_{\rm HI}$ distribution on IGM parameters trained on an ensemble of 624 Nyx hydrodynamical simulations at $z = 0.1$, which we combine with a Gaussian process emulator of the normalization. To demonstrate the efficacy of this approach, we generate hundreds of realizations of realistic mock HST/COS datasets, each comprising 34 quasar sightlines, and forward model the noise and resolution to match the real data. We use this large ensemble of mocks to extensively test our inference and empirically demonstrate that our posterior distributions are robust. Our analysis shows that by applying our new approach to existing Ly$α$ forest spectra at $z\simeq 0.1$, one can measure the thermal and ionization state of the IGM with very high precision ($σ_{\log T_0} \sim 0.08$ dex, $σ_γ\sim 0.06$, and $σ_{\log Γ_{\rm HI}} \sim 0.07$ dex).

astro-ph.CO↗