arXiv subjects

Haoran Li

Publications and source records attributed to Haoran Li.

At least 325 records · Page 18Linked to original sources

arXiv2021

High precision differential clock comparisons with a multiplexed optical lattice clock

Rapid progress in the precision and accuracy of optical atomic clocks over the last decade has advanced the frontiers of timekeeping, metrology, and quantum science. However, the stabilities of most optical clocks remain limited by the local oscillator rather than the atoms themselves, leaving room for further progress. Here we implement a "multiplexed" one-dimensional optical lattice clock, in which spatially-resolved, movable ensembles of ultra-cold strontium atoms are trapped in the same optical lattice, interrogated simultaneously by a shared clock laser, and read-out in parallel. By performing synchronized Ramsey interrogations of ensemble pairs we observe atom-atom coherence times up to 26 seconds, a 270-fold improvement over the atom-laser coherence time, demonstrate a relative stability of $9.7(4)\times10^{-18}/\sqrtτ$ (where $τ$ is the averaging time in seconds), and reach a fractional uncertainty of $8.9(3)\times 10^{-20}$ after 3.3 hours of averaging. These results demonstrate that applications requiring ultra-high-precision comparisons between optical atomic clocks need not be limited by the stability of the local oscillator. With multiple ensemble pairs, we realize a miniaturized clock network consisting of 6 atom ensembles, resulting in 15 unique pairwise clock comparisons with relative stabilities below $3\times10^{-17}/\sqrtτ$. Finally, we demonstrate the capability to simultaneously load spatially-resolved, heterogeneous ensemble pairs of all four stable isotopes of strontium in a lattice. The unique capabilities offered by this platform pave the way for future studies of precision isotope shift measurements, spatially resolved characterization of limiting clock systematics, development of clock-based gravitational wave and dark matter detectors, and novel tests of relativity including measurements of the gravitational redshift at sub-centimeter scales.

physics.atom-ph↗

arXiv2021

Data-driven turbulence modeling in separated flows considering physical mechanism analysis

Accurate simulation of turbulent flow with separation is an important but challenging problem. In this paper, a data-driven Reynolds-averaged turbulence modeling approach, field inversion and machine learning is implemented to modify the Spalart-Allmaras model separately on three cases, namely, the S809 airfoil, a periodic hill and the GLC305 airfoil with ice shape 944. Field inversion based on a discrete adjoint method is used to quantify the model-form uncertainty with limited experimental data. An artificial neural network is trained to predict the model corrections with local flow features to extract generalized modeling knowledge. Physical knowledge of the nonequilibrium turbulence in the separating shear layer is considered when setting the prior model uncertainty. The results show that the model corrections from the field inversion demonstrate strong consistency with the underlying physical mechanism of nonequilibrium turbulence. The quantity of interest from the observation data can be reproduced with relatively high accuracy by the augmented model. In addition, the validation in similar flow conditions shows a certain extent of generalization ability.

physics.flu-dyn↗

arXiv2021

Differentially Private Federated Knowledge Graphs Embedding

Knowledge graph embedding plays an important role in knowledge representation, reasoning, and data mining applications. However, for multiple cross-domain knowledge graphs, state-of-the-art embedding models cannot make full use of the data from different knowledge domains while preserving the privacy of exchanged data. In addition, the centralized embedding model may not scale to the extensive real-world knowledge graphs. Therefore, we propose a novel decentralized scalable learning framework, \emph{Federated Knowledge Graphs Embedding} (FKGE), where embeddings from different knowledge graphs can be learnt in an asynchronous and peer-to-peer manner while being privacy-preserving. FKGE exploits adversarial generation between pairs of knowledge graphs to translate identical entities and relations of different domains into near embedding spaces. In order to protect the privacy of the training data, FKGE further implements a privacy-preserving neural network structure to guarantee no raw data leakage. We conduct extensive experiments to evaluate FKGE on 11 knowledge graphs, demonstrating a significant and consistent improvement in model quality with at most 17.85\% and 7.90\% increases in performance on triple classification and link prediction tasks.

cs.LG↗

arXiv2021

Blind Image Quality Assessment for MRI with A Deep Three-dimensional content-adaptive Hyper-Network

Image Quality Assessment (IQA) is of great value in the workflow of Magnetic Resonance Imaging (MRI)-based analysis. Blind IQA (BIQA) methods are especially required since high-quality reference MRI images are usually not available. Recently, many efforts have been devoted to developing deep learning-based BIQA approaches. However, the performance of these methods is limited due to the utilization of simple content-non-adaptive network parameters and the waste of the important 3D spatial information of the medical images. To address these issues, we design a 3D content-adaptive hyper-network for MRI BIQA. The overall 3D configuration enables the exploration of comprehensive 3D spatial information from MRI images, while the developed content-adaptive hyper-network contributes to the self-adaptive capacity of network parameters and thus, facilitates better BIQA performance. The effectiveness of the proposed method is extensively evaluated on the open dataset, MRIQC. Promising performance is achieved compared with the corresponding baseline and 4 state-of-the-art BIQA methods. We make our code available at \url{https://git.openi.org.cn/SIAT_Wangshanshan/HyS-Net}.

eess.IV↗

arXiv2021

Mixed Cross Entropy Loss for Neural Machine Translation

In neural machine translation, cross entropy (CE) is the standard loss function in two training methods of auto-regressive models, i.e., teacher forcing and scheduled sampling. In this paper, we propose mixed cross entropy loss (mixed CE) as a substitute for CE in both training approaches. In teacher forcing, the model trained with CE regards the translation problem as a one-to-one mapping process, while in mixed CE this process can be relaxed to one-to-many. In scheduled sampling, we show that mixed CE has the potential to encourage the training and testing behaviours to be similar to each other, more effectively mitigating the exposure bias problem. We demonstrate the superiority of mixed CE over CE on several machine translation datasets, WMT'16 Ro-En, WMT'16 Ru-En, and WMT'14 En-De in both teacher forcing and scheduled sampling setups. Furthermore, in WMT'14 En-De, we also find mixed CE consistently outperforms CE on a multi-reference set as well as a challenging paraphrased reference set. We also found the model trained with mixed CE is able to provide a better probability distribution defined over the translation output space. Our code is available at https://github.com/haorannlp/mix.

cs.CL↗

arXiv2021

Syntax-augmented Multilingual BERT for Cross-lingual Transfer

In recent years, we have seen a colossal effort in pre-training multilingual text encoders using large-scale corpora in many languages to facilitate cross-lingual transfer learning. However, due to typological differences across languages, the cross-lingual transfer is challenging. Nevertheless, language syntax, e.g., syntactic dependencies, can bridge the typological gap. Previous works have shown that pre-trained multilingual encoders, such as mBERT \cite{devlin-etal-2019-bert}, capture language syntax, helping cross-lingual transfer. This work shows that explicitly providing language syntax and training mBERT using an auxiliary objective to encode the universal dependency tree structure helps cross-lingual transfer. We perform rigorous experiments on four NLP tasks, including text classification, question answering, named entity recognition, and task-oriented semantic parsing. The experiment results show that syntax-augmented mBERT improves cross-lingual transfer on popular benchmarks, such as PAWS-X and MLQA, by 1.4 and 1.6 points on average across all languages. In the \emph{generalized} transfer setting, the performance boosted significantly, with 3.9 and 3.1 points on average in PAWS-X and MLQA.

cs.CL↗

arXiv2021

Dynamic radiomics: a new methodology to extract quantitative time-related features from tomographic images

The feature extraction methods of radiomics are mainly based on static tomographic images at a certain moment, while the occurrence and development of disease is a dynamic process that cannot be fully reflected by only static characteristics. This study proposes a new dynamic radiomics feature extraction workflow that uses time-dependent tomographic images of the same patient, focuses on the changes in image features over time, and then quantifies them as new dynamic features for diagnostic or prognostic evaluation. We first define the mathematical paradigm of dynamic radiomics and introduce three specific methods that can describe the transformation process of features over time. Three different clinical problems are used to validate the performance of the proposed dynamic feature with conventional 2D and 3D static features.

eess.IV↗

arXiv2021

ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining

While online conversations can cover a vast amount of information in many different formats, abstractive text summarization has primarily focused on modeling solely news articles. This research gap is due, in part, to the lack of standardized datasets for summarizing online discussions. To address this gap, we design annotation protocols motivated by an issues--viewpoints--assertions framework to crowdsource four new datasets on diverse online conversation forms of news comments, discussion forums, community question answering forums, and email threads. We benchmark state-of-the-art models on our datasets and analyze characteristics associated with the data. To create a comprehensive benchmark, we also evaluate these models on widely-used conversation summarization datasets to establish strong baselines in this domain. Furthermore, we incorporate argument mining through graph construction to directly model the issues, viewpoints, and assertions present in a conversation and filter noisy input, showing comparable or improved results according to automatic and human evaluations.

cs.CL↗

arXiv2021

Aerodynamic Prediction of High-Lift Configuration Using k-(v^2 )-ω Turbulence Model

The aerodynamic performance of the high-lift configuration greatly influences the safety and economy of commercial aircraft. Accurately predicting the aerodynamic performance of the high-lift configuration, especially the stall behavior, is important for aircraft design. However, the complex flow phenomena of high-lift configurations pose substantial difficulties to current turbulence models. In this paper, a three-equation k-(v^2)-ω turbulence model for the Reynolds-averaged Navier-Stokes equations is used to compute the stall behavior of high-lift configurations. A separated shear layer fixed function is implemented in the turbulence model to better capture the nonequilibrium characteristics of turbulence. Different high-lift configurations, including the two-dimensional multielement NLR7301 and Omar airfoils and a complex full-configuration model (JAXA Standard Model), are numerically tested. The results indicate that the effect of the nonequilibrium characteristics of turbulence is significant in the free shear layer, which is key to accurately predicting the stall behavior of high-lift devices. The modified SPF k-(v^2 )-ω model is more accurate in predicting stall behavior than the Spalart-Allmaras, shear stress transport, and original k-(v^2)-ω models for the full high-lift configuration. The relative errors in the predicted maximum lift coefficients are within 3% of the experimental data.

physics.flu-dyn↗

arXiv2021

EASE: Extractive-Abstractive Summarization with Explanations

Current abstractive summarization systems outperform their extractive counterparts, but their widespread adoption is inhibited by the inherent lack of interpretability. To achieve the best of both worlds, we propose EASE, an extractive-abstractive framework for evidence-based text generation and apply it to document summarization. We present an explainable summarization system based on the Information Bottleneck principle that is jointly trained for extraction and abstraction in an end-to-end fashion. Inspired by previous research that humans use a two-stage framework to summarize long documents (Jing and McKeown, 2000), our framework first extracts a pre-defined amount of evidence spans as explanations and then generates a summary using only the evidence. Using automatic and human evaluations, we show that explanations from our framework are more relevant than simple baselines, without substantially sacrificing the quality of the generated summary.

cs.CL↗

arXiv2021

Lifelong Learning with Sketched Structural Regularization

Preventing catastrophic forgetting while continually learning new tasks is an essential problem in lifelong learning. Structural regularization (SR) refers to a family of algorithms that mitigate catastrophic forgetting by penalizing the network for changing its "critical parameters" from previous tasks while learning a new one. The penalty is often induced via a quadratic regularizer defined by an \emph{importance matrix}, e.g., the (empirical) Fisher information matrix in the Elastic Weight Consolidation framework. In practice and due to computational constraints, most SR methods crudely approximate the importance matrix by its diagonal. In this paper, we propose \emph{Sketched Structural Regularization} (Sketched SR) as an alternative approach to compress the importance matrices used for regularizing in SR methods. Specifically, we apply \emph{linear sketching methods} to better approximate the importance matrices in SR algorithms. We show that sketched SR: (i) is computationally efficient and straightforward to implement, (ii) provides an approximation error that is justified in theory, and (iii) is method oblivious by construction and can be adapted to any method that belongs to the structural regularization class. We show that our proposed approach consistently improves various SR algorithms' performance on both synthetic experiments and benchmark continual learning tasks, including permuted-MNIST and CIFAR-100.

cs.LG↗

arXiv2021

Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation

Models pretrained with self-supervised objectives on large text corpora achieve state-of-the-art performance on English text summarization tasks. However, these models are typically fine-tuned on hundreds of thousands of data points, an infeasible requirement when applying summarization to new, niche domains. In this work, we introduce a novel and generalizable method, called WikiTransfer, for fine-tuning pretrained models for summarization in an unsupervised, dataset-specific manner. WikiTransfer fine-tunes pretrained models on pseudo-summaries, produced from generic Wikipedia data, which contain characteristics of the target dataset, such as the length and level of abstraction of the desired summaries. WikiTransfer models achieve state-of-the-art, zero-shot abstractive summarization performance on the CNN-DailyMail dataset and demonstrate the effectiveness of our approach on three additional diverse datasets. These models are more robust to noisy data and also achieve better or comparable few-shot performance using 10 and 100 training examples when compared to few-shot transfer from other summarization datasets. To further boost performance, we employ data augmentation via round-trip translation as well as introduce a regularization term for improved few-shot transfer. To understand the role of dataset aspects in transfer performance and the quality of the resulting output summaries, we further study the effect of the components of our unsupervised fine-tuning data and analyze few-shot performance using both automatic and human evaluation.

cs.CL↗

arXiv2021

Numerical Simulation of Iced Wing Using Separating Shear Layer Fixed Turbulence Models

Aerodynamic prediction of glaze ice accretion on airfoils and wing is studied using the Reynolds-averaged Navier-Stokes method. Two separation fixed turbulence models are developed considering the nonequilibrium characteristics of turbulence. The key ad hoc fix is a term of the local ratio of turbulent production to dissipation, which is used to amplify the destruction term of the ω-equation to increase the eddy viscosity in a separating shear layer of the fully turbulent region. A shear stress limiter is adopted to appropriately simulate the beginning process of the shear layer transition when the turbulence is under development. The proposed separation fixed terms can be easily implemented into current solvers. Two airfoils and a three-dimensional swept wing with ice accretions are numerically tested using the modified models. The results indicate that the separating shear layer fixes improve the ability of the models in predicting the stall behavior at large angles of attack. The simulated averaged flow field and turbulence intensity distribution are consistent with experimental data.

physics.flu-dyn↗

arXiv2021

MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmark

Scaling semantic parsing models for task-oriented dialog systems to new languages is often expensive and time-consuming due to the lack of available datasets. Available datasets suffer from several shortcomings: a) they contain few languages b) they contain small amounts of labeled examples per language c) they are based on the simple intent and slot detection paradigm for non-compositional queries. In this paper, we present a new multilingual dataset, called MTOP, comprising of 100k annotated utterances in 6 languages across 11 domains. We use this dataset and other publicly available datasets to conduct a comprehensive benchmarking study on using various state-of-the-art multilingual pre-trained models for task-oriented semantic parsing. We achieve an average improvement of +6.3 points on Slot F1 for the two existing multilingual datasets, over best results reported in their experiments. Furthermore, we demonstrate strong zero-shot performance using pre-trained models combined with automatic translation and alignment, and a proposed distant supervision method to reduce the noise in slot label projection.

cs.CL↗

arXiv2020

Conversational Semantic Parsing

The structured representation for semantic parsing in task-oriented assistant systems is geared towards simple understanding of one-turn queries. Due to the limitations of the representation, the session-based properties such as co-reference resolution and context carryover are processed downstream in a pipelined system. In this paper, we propose a semantic representation for such task-oriented conversational systems that can represent concepts such as co-reference and context carryover, enabling comprehensive understanding of queries in a session. We release a new session-based, compositional task-oriented parsing dataset of 20k sessions consisting of 60k utterances. Unlike Dialog State Tracking Challenges, the queries in the dataset have compositional forms. We propose a new family of Seq2Seq models for the session-based parsing above, which achieve better or comparable performance to the current state-of-the-art on ATIS, SNIPS, TOP and DSTC2. Notably, we improve the best known results on DSTC2 by up to 5 points for slot-carryover.

cs.CL↗

arXiv2020

Multimodal Joint Attribute Prediction and Value Extraction for E-commerce Product

Product attribute values are essential in many e-commerce scenarios, such as customer service robots, product recommendations, and product retrieval. While in the real world, the attribute values of a product are usually incomplete and vary over time, which greatly hinders the practical applications. In this paper, we propose a multimodal method to jointly predict product attributes and extract values from textual product descriptions with the help of the product images. We argue that product attributes and values are highly correlated, e.g., it will be easier to extract the values on condition that the product attributes are given. Thus, we jointly model the attribute prediction and value extraction tasks from multiple aspects towards the interactions between attributes and values. Moreover, product images have distinct effects on our tasks for different product attributes and values. Thus, we selectively draw useful visual information from product images to enhance our model. We annotate a multimodal product attribute value dataset that contains 87,194 instances, and the experimental results on this dataset demonstrate that explicitly modeling the relationship between attributes and values facilitates our method to establish the correspondence between them, and selectively utilizing visual product information is necessary for the task. Our code and dataset will be released to the public.

cs.CL↗

arXiv2020

Deep Reinforcement Learning based Automatic Exploration for Navigation in Unknown Environment

This paper investigates the automatic exploration problem under the unknown environment, which is the key point of applying the robotic system to some social tasks. The solution to this problem via stacking decision rules is impossible to cover various environments and sensor properties. Learning based control methods are adaptive for these scenarios. However, these methods are damaged by low learning efficiency and awkward transferability from simulation to reality. In this paper, we construct a general exploration framework via decomposing the exploration process into the decision, planning, and mapping modules, which increases the modularity of the robotic system. Based on this framework, we propose a deep reinforcement learning based decision algorithm which uses a deep neural network to learning exploration strategy from the partial map. The results show that this proposed algorithm has better learning efficiency and adaptability for unknown environments. In addition, we conduct the experiments on the physical robot, and the results suggest that the learned policy can be well transfered from simulation to the real robot.

cs.RO↗

arXiv2020

Emerging Cross-lingual Structure in Pretrained Language Models

We study the problem of multilingual masked language modeling, i.e. the training of a single model on concatenated text from multiple languages, and present a detailed study of several factors that influence why these models are so effective for cross-lingual transfer. We show, contrary to what was previously hypothesized, that transfer is possible even when there is no shared vocabulary across the monolingual corpora and also when the text comes from very different domains. The only requirement is that there are some shared parameters in the top layers of the multi-lingual encoder. To better understand this result, we also show that representations from independently trained models in different languages can be aligned post-hoc quite effectively, strongly suggesting that, much like for non-contextual word embeddings, there are universal latent symmetries in the learned embedding spaces. For multilingual masked language modeling, these symmetries seem to be automatically discovered and aligned during the joint training process.

cs.CL↗