arXiv subjects

Hao Tian

Publications and source records attributed to Hao Tian.

At least 145 records · Page 8Linked to original sources

arXiv2022

Indicative Image Retrieval: Turning Blackbox Learning into Grey

Deep learning became the game changer for image retrieval soon after it was introduced. It promotes the feature extraction (by representation learning) as the core of image retrieval, with the relevance/matching evaluation being degenerated into simple similarity metrics. In many applications, we need the matching evidence to be indicated rather than just have the ranked list (e.g., the locations of the target proteins/cells/lesions in medical images). It is like the matched words need to be highlighted in search engines. However, this is not easy to implement without explicit relevance/matching modeling. The deep representation learning models are not feasible because of their blackbox nature. In this paper, we revisit the importance of relevance/matching modeling in deep learning era with an indicative retrieval setting. The study shows that it is possible to skip the representation learning and model the matching evidence directly. By removing the dependency on the pre-trained models, it has avoided a lot of related issues (e.g., the domain gap between classification and retrieval, the detail-diffusion caused by convolution, and so on). More importantly, the study demonstrates that the matching can be explicitly modeled and backtracked later for generating the matching evidence indications. It can improve the explainability of deep inference. Our method obtains a best performance in literature on both Oxford-5k and Paris-6k, and sets a new record of 97.77% on Oxford-5k (97.81% on Paris-6k) without extracting any deep features.

cs.CV↗

arXiv2022

Snowmass 2021 White Paper: The Windchime Project

The absence of clear signals from particle dark matter in direct detection experiments motivates new approaches in disparate regions of viable parameter space. In this Snowmass white paper, we outline the Windchime project, a program to build a large array of quantum-enhanced mechanical sensors. The ultimate aim is to build a detector capable of searching for Planck mass-scale dark matter purely through its gravitational coupling to ordinary matter. In the shorter term, we aim to search for a number of other physics targets, especially some ultralight dark matter candidates. Here, we discuss the basic design, open R&D challenges and opportunities, current experimental efforts, and both short- and long-term physics targets of the Windchime project.

hep-ex↗

arXiv2021

ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generation

Conventional methods for the image-text generation tasks mainly tackle the naturally bidirectional generation tasks separately, focusing on designing task-specific frameworks to improve the quality and fidelity of the generated samples. Recently, Vision-Language Pre-training models have greatly improved the performance of the image-to-text generation tasks, but large-scale pre-training models for text-to-image synthesis task are still under-developed. In this paper, we propose ERNIE-ViLG, a unified generative pre-training framework for bidirectional image-text generation with transformer model. Based on the image quantization models, we formulate both image generation and text generation as autoregressive generative tasks conditioned on the text/image input. The bidirectional image-text generative modeling eases the semantic alignments across vision and language. For the text-to-image generation process, we further propose an end-to-end training method to jointly learn the visual sequence generator and the image reconstructor. To explore the landscape of large-scale pre-training for bidirectional text-image generation, we train a 10-billion parameter ERNIE-ViLG model on a large-scale dataset of 145 million (Chinese) image-text pairs which achieves state-of-the-art performance for both text-to-image and image-to-text tasks, obtaining an FID of 7.9 on MS-COCO for text-to-image synthesis and best results on COCO-CN and AIC-ICC for image captioning.

cs.CV↗

arXiv2021

ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

Pre-trained language models have achieved state-of-the-art results in various Natural Language Processing (NLP) tasks. GPT-3 has shown that scaling up pre-trained language models can further exploit their enormous potential. A unified framework named ERNIE 3.0 was recently proposed for pre-training large-scale knowledge enhanced models and trained a model with 10 billion parameters. ERNIE 3.0 outperformed the state-of-the-art models on various NLP tasks. In order to explore the performance of scaling up ERNIE 3.0, we train a hundred-billion-parameter model called ERNIE 3.0 Titan with up to 260 billion parameters on the PaddlePaddle platform. Furthermore, we design a self-supervised adversarial loss and a controllable language modeling loss to make ERNIE 3.0 Titan generate credible and controllable texts. To reduce the computation overhead and carbon emission, we propose an online distillation framework for ERNIE 3.0 Titan, where the teacher model will teach students and train itself simultaneously. ERNIE 3.0 Titan is the largest Chinese dense pre-trained model so far. Empirical results show that the ERNIE 3.0 Titan outperforms the state-of-the-art models on 68 NLP datasets.

cs.CL↗

arXiv2021

The Spectroscopic Binaries from LAMOST Medium-Resolution Survey (MRS). I. Searching for Double-lined Spectroscopic Binaries (SB2s) with Convolutional Neural Network

We developed a convolutional neural network (CNN) model to distinguish the double-lined spectroscopic binaries (SB2s) from others based on single exposure medium-resolution spectra ($R\sim 7,500$). The training set consists of a large set of mock spectra of single stars and binaries synthesized based on the MIST stellar evolutionary model and ATLAS9 atmospheric model. Our model reaches a novel theoretic false positive rate by adding a proper penalty on the negative sample (e.g., 0.12\% and 0.16\% for the blue/red arm when the penalty parameter $Λ=16$). Tests show that the performance is as expected and favors FGK-type Main-sequence binaries with high mass ratio ($q \geq 0.7$) and large radial velocity separation ($Δv \geq 50\,\mathrm{km\,s^{-1}}$). Although the real false positive rate can not be estimated reliably, validating on eclipsing binaries identified from Kepler light curves indicates that our model predicts low binary probabilities at eclipsing phases (0, 0.5, and 1.0) as expected. The color-magnitude diagram also helps illustrate its feasibility and capability of identifying FGK MS binaries from spectra. We conclude that this model is reasonably reliable and can provide an automatic approach to identify SB2s with period $\lesssim 10$ days. This work yields a catalog of binary probabilities for over 5 million spectra of 1 million sources from the LAMOST medium-resolution survey (MRS), and a catalog of 2198 SB2 candidates whose physical properties will be analyzed in our following-up paper. Data products are made publicly available at the journal as well as our Github website.

astro-ph.SR↗

arXiv2021

The Stellar "Snake" I: Whole Structure and Properties

To complement our previous discovery of the young snake-like structure in the solar neighborhood and reveal the structure's full extent, we build two samples of stars within the Snake and its surrounding territory from {\tt Gaia EDR3}. With the friends-of-friends algorithm, we identify 2694 and 9615 Snake member candidates from the two samples. Thirteen open clusters are embedded in these member candidates. By combining the spectroscopic data from multiple surveys, we investigate the comprehensive properties of the candidates and find that they \thj{are very likely to} belong to one sizable structure, since most of the components are well bridged in their spatial distributions, and follow a single stellar population with an age of $30-40$\,Myr and solar metallicity. This sizable structure is best explained as hierarchically primordial, and probably formed from a filamentary giant molecular cloud with unique formation history in localized regions. To analyze the dynamics of the Snake, we divide the structure into five groups according to their tangential velocities; we find that the groups are expanding at a coherent rate ($κ_X\sim3.0\,\times10^{-2}\,\rm km\,s^{-1}\,pc^{-1}$) along the length of the structure ($X$-direction). \thj{The corresponding expansion age ($τ\sim33$\,Myr) is highly consistent with the age of the Snake}. With over ten thousand member stars, the Snake is an ideal laboratory to study nearby coeval stellar formation, stellar physics, and environmental evolution over a large spatial extent.

astro-ph.GA↗

arXiv2021

Quantum coherent microwave-optical transduction using high overtone bulk acoustic resonances

A device capable of converting single quanta of the microwave field to the optical domain is an outstanding endeavour in the context of quantum interconnects between distant superconducting qubits, but likewise can have applications in other fields, such as radio astronomy or, in the classical realm, microwave photonics. A variety of transduction approaches, based on optomechanical or electro-optical interactions, have been proposed and realized, yet the required vanishing added noises and an efficiency approaching unity, have not yet been attained. Here we present a new transduction scheme that could in theory satisfy the requirements for quantum coherent bidirectional transduction. Our scheme relies on an intermediary mechanical mode, a high overtone bulk acoustic resonance (HBAR), to coherently couple microwave and optical photons through the piezoelectric and strain-optical effects. Its efficiency results from the combination of integrated Si3N4 photonic circuits with ultra low loss sustaining high intracavity photon numbers with the highly efficient microwave to mechanical transduction offered by piezoelectrically coupled HBAR. We develop a quantum theory for this multipartite system by first introducing a quantization method for the piezoelectric interaction between the microwave mode and the mechanical mode from first principles (which to our knowledge has not been presented in this form), and link the latter to the conventional Butterworth-Van Dyke model. The HBAR is subsequently coupled to a pair of hybridized optical modes from coupled optical ring cavities via the strain-optical effect. We analyze the conversion capabilities of the proposed device using signal flow graphs, and demonstrate that near quantum coherent transduction is possible, with realistic experimental parameters.

quant-ph↗

arXiv2021

ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora

Recent studies have demonstrated that pre-trained cross-lingual models achieve impressive performance in downstream cross-lingual tasks. This improvement benefits from learning a large amount of monolingual and parallel corpora. Although it is generally acknowledged that parallel corpora are critical for improving the model performance, existing methods are often constrained by the size of parallel corpora, especially for low-resource languages. In this paper, we propose ERNIE-M, a new training method that encourages the model to align the representation of multiple languages with monolingual corpora, to overcome the constraint that the parallel corpus size places on the model performance. Our key insight is to integrate back-translation into the pre-training process. We generate pseudo-parallel sentence pairs on a monolingual corpus to enable the learning of semantic alignments between different languages, thereby enhancing the semantic modeling of cross-lingual models. Experimental results show that ERNIE-M outperforms existing cross-lingual models and delivers new state-of-the-art results in various cross-lingual downstream tasks.

cs.CL↗

arXiv2021

Reinforcement Learning with Evolutionary Trajectory Generator: A General Approach for Quadrupedal Locomotion

Recently reinforcement learning (RL) has emerged as a promising approach for quadrupedal locomotion, which can save the manual effort in conventional approaches such as designing skill-specific controllers. However, due to the complex nonlinear dynamics in quadrupedal robots and reward sparsity, it is still difficult for RL to learn effective gaits from scratch, especially in challenging tasks such as walking over the balance beam. To alleviate such difficulty, we propose a novel RL-based approach that contains an evolutionary foot trajectory generator. Unlike prior methods that use a fixed trajectory generator, the generator continually optimizes the shape of the output trajectory for the given task, providing diversified motion priors to guide the policy learning. The policy is trained with reinforcement learning to output residual control signals that fit different gaits. We then optimize the trajectory generator and policy network alternatively to stabilize the training and share the exploratory data to improve sample efficiency. As a result, our approach can solve a range of challenging tasks in simulation by learning from scratch, including walking on a balance beam and crawling through the cave. To further verify the effectiveness of our approach, we deploy the controller learned in the simulation on a 12-DoF quadrupedal robot, and it can successfully traverse challenging scenarios with efficient gaits.

cs.RO↗

arXiv2021

ADER:Adapting between Exploration and Robustness for Actor-Critic Methods

Combining off-policy reinforcement learning methods with function approximators such as neural networks has been found to lead to overestimation of the value function and sub-optimal solutions. Improvement such as TD3 has been proposed to address this issue. However, we surprisingly find that its performance lags behind the vanilla actor-critic methods (such as DDPG) in some primitive environments. In this paper, we show that the failure of some cases can be attributed to insufficient exploration. We reveal the culprit of insufficient exploration in TD3, and propose a novel algorithm toward this problem that ADapts between Exploration and Robustness, namely ADER. To enhance the exploration ability while eliminating the overestimation bias, we introduce a dynamic penalty term in value estimation calculated from estimated uncertainty, which takes into account different compositions of the uncertainty in different learning stages. Experiments in several challenging environments demonstrate the supremacy of the proposed method in continuous control tasks.

cs.LG↗

arXiv2021

LAMOST Time-Domain Survey: First Results of four $K$2 plates

From Oct. 2019 to Apr. 2020, LAMOST performs a time-domain spectroscopic survey of four $K$2 plates with both low- and med-resolution observations. The low-resolution spectroscopic survey gains 282 exposures ($\approx$46.6 hours) over 25 nights, yielding a total of about 767,000 spectra, and the med-resolution survey takes 177 exposures ($\approx$49.1 hours) over 27 nights, collecting about 478,000 spectra. More than 70%/50% of low-resolution/med-resolution spectra have signal-to-noise ratio higher than 10. We determine stellar parameters (e.g., $T_{\rm eff}$, log$g$, [Fe/H]) and radial velocity (RV) with different methods, including LASP, DD-Payne, and SLAM. In general, these parameter estimations from different methods show good agreement, and the stellar parameter values are consistent with those of APOGEE. We use the $Gaia$ DR2 RV data to calculate a median RV zero point (RVZP) for each spectrograph exposure by exposure, and the RVZP-corrected RVs agree well with the APOGEE data. The stellar evolutionary and spectroscopic masses are estimated based on the stellar parameters, multi-band magnitudes, distances and extinction values. Finally, we construct a binary catalog including about 2700 candidates by analyzing their light curves, fitting the RV data, calculating the binarity parameters from med-resolution spectra, and cross-matching the spatially resolved binary catalog from $Gaia$ EDR3. The LAMOST TD survey is expected to get breakthrough in various scientific topics, such as binary system, stellar activity, and stellar pulsation, etc.

astro-ph.SR↗

arXiv2021

The Flare and Warp of the Young Stellar Disk traced with LAMOST DR5 OB-type stars

We present analysis of the spatial density structure for the outer disk from 8$-$14 \,kpc with the LAMOST DR5 13534 OB-type stars and observe similar flaring on north and south sides of the disk implying that the flaring structure is symmetrical about the Galactic plane, for which the scale height at different Galactocentric distance is from 0.14 to 0.5 \,kpc. By using the average slope to characterize the flaring strength we find that the thickness of the OB stellar disk is similar but flaring is slightly stronger compared to the thin disk as traced by red giant branch stars, possibly implying that secular evolution is not the main contributor to the flaring but perturbation scenarios such as interactions with passing dwarf galaxies should be more possible. When comparing the scale height of OB stellar disk of the north and south sides with the gas disk, the former one is slightly thicker than the later one by $\approx$ 33 and 9 \,pc, meaning that one could tentatively use young OB-type stars to trace the gas properties. Meanwhile, we unravel that the radial scale length of the young OB stellar disk is 1.17 $\pm$ 0.05 \,kpc, which is shorter than that of the gas disk, confirming that the gas disk is more extended than stellar disk. What is more, by considering the mid-plane displacements ($Z_{0}$) in our density model we find that almost all of $Z_{0}$ are within 100 \,pc with the increasing trend as Galactocentric distance increases.

astro-ph.GA↗

arXiv2021

Ultralow-noise frequency-agile photonic integrated lasers

Low-noise lasers are of central importance in a wide variety of applications, including high spectral-efficiency coherent communication protocols, distributed fibre sensing, and long distance coherent LiDAR. In addition to low phase noise, frequency agility, that is, the ability to achieve high-bandwidth actuation of the laser frequency, is imperative for triangular chirping in frequency-modulated continuous-wave (FMCW) based ranging or any optical phase locking as routinely used in metrology. While integrated silicon-based lasers have experienced major advances and are now employed on a commercial scale in data centers, integrated lasers with sub-100 Hz-level intrinsic linewidth are based on optical feedback from photonic circuits that lack frequency agility. Here, we demonstrate a wafer-scale-manufacturing-compatible hybrid photonic integrated laser that exhibits ultralow intrinsic linewidth of 25 Hz while offering unsurpassed megahertz actuation bandwidth, with a tuning range larger than 1 GHz. Our approach uses ultralow-loss (1 dB/m) Si$_3$N$_4$ photonic microresonators, combined with aluminium nitride (AlN) or lead zirconium titanate (PZT) microelectromechanical systems (MEMS) based stress-optic actuation. Electrically driven low-phase noise lasing is attained by self-injection locking of an Indium Phosphide (InP) laser chip and only limited by fundamental thermo-refractive noise. By utilizing difference drive and apodization of the photonic chip, a flat actuation response up to 10 MHz is achieved. We leverage this capability to demonstrate a compact coherent LiDAR engine that can generate up to 800 kHz FMCW triangular optical chirp signals, requiring neither any active linearization nor predistortion compensation, and perform a 10 m optical ranging experiment, with a resolution of 12.5 cm.

physics.optics↗

arXiv2021

ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

Pre-trained models have achieved state-of-the-art results in various Natural Language Processing (NLP) tasks. Recent works such as T5 and GPT-3 have shown that scaling up pre-trained language models can improve their generalization abilities. Particularly, the GPT-3 model with 175 billion parameters shows its strong task-agnostic zero-shot/few-shot learning capabilities. Despite their success, these large-scale models are trained on plain texts without introducing knowledge such as linguistic knowledge and world knowledge. In addition, most large-scale models are trained in an auto-regressive way. As a result, this kind of traditional fine-tuning approach demonstrates relatively weak performance when solving downstream language understanding tasks. In order to solve the above problems, we propose a unified framework named ERNIE 3.0 for pre-training large-scale knowledge enhanced models. It fuses auto-regressive network and auto-encoding network, so that the trained model can be easily tailored for both natural language understanding and generation tasks with zero-shot learning, few-shot learning or fine-tuning. We trained the model with 10 billion parameters on a 4TB corpus consisting of plain texts and a large-scale knowledge graph. Empirical results show that the model outperforms the state-of-the-art models on 54 Chinese NLP tasks, and its English version achieves the first place on the SuperGLUE benchmark (July 3, 2021), surpassing the human performance by +0.8% (90.6% vs. 89.8%).

cs.CL↗

arXiv2021

Action Set Based Policy Optimization for Safe Power Grid Management

Maintaining the stability of the modern power grid is becoming increasingly difficult due to fluctuating power consumption, unstable power supply coming from renewable energies, and unpredictable accidents such as man-made and natural disasters. As the operation on the power grid must consider its impact on future stability, reinforcement learning (RL) has been employed to provide sequential decision-making in power grid management. However, existing methods have not considered the environmental constraints. As a result, the learned policy has risk of selecting actions that violate the constraints in emergencies, which will escalate the issue of overloaded power lines and lead to large-scale blackouts. In this work, we propose a novel method for this problem, which builds on top of the search-based planning algorithm. At the planning stage, the search space is limited to the action set produced by the policy. The selected action strictly follows the constraints by testing its outcome with the simulation function provided by the system. At the learning stage, to address the problem that gradients cannot be propagated to the policy, we introduce Evolutionary Strategies (ES) with black-box policy optimization to improve the policy directly, maximizing the returns of the long run. In NeurIPS 2020 Learning to Run Power Network (L2RPN) competition, our solution safely managed the power grid and ranked first in both tracks.

cs.AI↗

arXiv2021

ERNIE-Tiny : A Progressive Distillation Framework for Pretrained Transformer Compression

Pretrained language models (PLMs) such as BERT adopt a training paradigm which first pretrain the model in general data and then finetune the model on task-specific data, and have recently achieved great success. However, PLMs are notorious for their enormous parameters and hard to be deployed on real-life applications. Knowledge distillation has been prevailing to address this problem by transferring knowledge from a large teacher to a much smaller student over a set of data. We argue that the selection of thee three key components, namely teacher, training data, and learning objective, is crucial to the effectiveness of distillation. We, therefore, propose a four-stage progressive distillation framework ERNIE-Tiny to compress PLM, which varies the three components gradually from general level to task-specific level. Specifically, the first stage, General Distillation, performs distillation with guidance from pretrained teacher, gerenal data and latent distillation loss. Then, General-Enhanced Distillation changes teacher model from pretrained teacher to finetuned teacher. After that, Task-Adaptive Distillation shifts training data from general data to task-specific data. In the end, Task-Specific Distillation, adds two additional losses, namely Soft-Label and Hard-Label loss onto the last stage. Empirical results demonstrate the effectiveness of our framework and generalization gain brought by ERNIE-Tiny.In particular, experiments show that a 4-layer ERNIE-Tiny maintains over 98.0%performance of its 12-layer teacher BERT base on GLUE benchmark, surpassing state-of-the-art (SOTA) by 1.0% GLUE score with the same amount of parameters. Moreover, ERNIE-Tiny achieves a new compression SOTA on five Chinese NLP tasks, outperforming BERT base by 0.4% accuracy with 7.5x fewer parameters and9.4x faster inference speed.

cs.CL↗

arXiv2021

Self-consistent Stellar Radial Velocities from LAMOST Medium-Resolution Survey (MRS) DR7

Radial velocity (RV) is among the most fundamental physical quantities obtainable from stellar spectra and is rather important in the analysis of time-domain phenomena. The LAMOST Medium-Resolution Survey (MRS) DR7 contains 5 million single-exposure stellar spectra at spectral resolution $R\sim7\,500$. However, the temporal variation of the RV zero-points (RVZPs) of the MRS survey, which makes the RVs from multiple epochs inconsistent, has not been addressed. In this paper, we measure the RVs of the 3.8 million single-exposure spectra (for 0.6 million stars) with signal-to-noise ratio (SNR) higher than 5 based on cross-correlation function (CCF) method, and propose a robust method to self-consistently determine the RVZPs exposure-by-exposure for each spectrograph with the help of \textit{Gaia} DR2 RVs. Such RVZPs are estimated for 3.6 million RVs and can reach a mean precision of $\sim 0.38\,\mathrm{km\,s}^{-1}$. The result of the temporal variation of RVZPs indicates that our algorithm is efficient and necessary before we use the absolute RVs to perform time-domain analysis. Validating the results with APOGEE DR16 shows that our absolute RVs can reach an overall precision of 0.84/0.80 $\mathrm{km\,s}^{-1}$ in the blue/red arm at $50<\mathrm{SNR}<100$, while 1.26/1.99 $\mathrm{km\,s}^{-1}$ at $5<\mathrm{SNR}<10$. The cumulative distribution function (CDF) of the standard deviations of multiple RVs ($N_\mathrm{obs}\geq 8$) for 678 standard stars reach 0.45/0.54, 1.07/1.39, and 1.45/1.86 $\mathrm{km\,s}^{-1}$ in the blue/red arm at 50\%, 90\%, and 95\% levels, respectively. The catalogs of the RVs, RVZPs, and selected candidate RV standard stars are available at \url{https://github.com/hypergravity/paperdata}.

astro-ph.SR↗

arXiv2021

ERNIE-Doc: A Retrospective Long-Document Modeling Transformer

Transformers are not suited for processing long documents, due to their quadratically increasing memory and time consumption. Simply truncating a long document or applying the sparse attention mechanism will incur the context fragmentation problem or lead to an inferior modeling capability against comparable model sizes. In this paper, we propose ERNIE-Doc, a document-level language pretraining model based on Recurrence Transformers. Two well-designed techniques, namely the retrospective feed mechanism and the enhanced recurrence mechanism, enable ERNIE-Doc, which has a much longer effective context length, to capture the contextual information of a complete document. We pretrain ERNIE-Doc to explicitly learn the relationships among segments with an additional document-aware segment-reordering objective. Various experiments were conducted on both English and Chinese document-level tasks. ERNIE-Doc improved the state-of-the-art language modeling result of perplexity to 16.8 on WikiText-103. Moreover, it outperformed competitive pretraining models by a large margin on most language understanding tasks, such as text classification and question answering.

cs.CL↗