arXiv subjects

Xu Han

Publications and source records attributed to Xu Han.

At least 271 records · Page 15Linked to original sources

arXiv2022

Quantum transduction with microwave and optical entanglement

Quantum transduction refers to the coherent conversion between microwave and optical states, which can be achieved by quantum teleportation if given high fidelity microwave-optical entanglement, namely entanglement-based quantum transduction. Reliable microwave-optical entanglement can be generated using various platforms. In this paper, we base the discussion on piezo-optomechanical system and make the teleportation induced conversion scheme more concrete in the framework of quantum channel theory. By comparing the quantum capacity between the entanglement-based conversion channel and the traditional direct quantum transduction channel, we show entanglement-based scheme indeed admits a positive transduction rate when the direct quantum transduction has zero quantum capacity. Given two piezo-optomechanical systems, we also investigate the generation of microwave-microwave entanglement from entanglement swapping within continuous variable and discrete variable settings, showing the potentials of directly connecting microwave quantum processor by microwave-microwave quantum teleportation.

quant-ph↗

arXiv2021

Decoupled Low-light Image Enhancement

The visual quality of photographs taken under imperfect lightness conditions can be degenerated by multiple factors, e.g., low lightness, imaging noise, color distortion and so on. Current low-light image enhancement models focus on the improvement of low lightness only, or simply deal with all the degeneration factors as a whole, therefore leading to a sub-optimal performance. In this paper, we propose to decouple the enhancement model into two sequential stages. The first stage focuses on improving the scene visibility based on a pixel-wise non-linear mapping. The second stage focuses on improving the appearance fidelity by suppressing the rest degeneration factors. The decoupled model facilitates the enhancement in two aspects. On the one hand, the whole low-light enhancement can be divided into two easier subtasks. The first one only aims to enhance the visibility. It also helps to bridge the large intensity gap between the low-light and normal-light images. In this way, the second subtask can be shaped as the local appearance adjustment. On the other hand, since the parameter matrix learned from the first stage is aware of the lightness distribution and the scene structure, it can be incorporated into the second stage as the complementary information. In the experiments, our model demonstrates the state-of-the-art performance in both qualitative and quantitative comparisons, compared with other low-light image enhancement models. In addition, the ablation studies also validate the effectiveness of our model in multiple aspects, such as model structure and loss function. The trained model is available at https://github.com/hanxuhfut/Decoupled-Low-light-Image-Enhancement.

cs.CV↗

arXiv2021

Atomistic View of Homogeneous Nucleation of Water into Polymorphic Ices

Water is one of the most abundant substances on Earth, and ice, i.e., solid water, has more than 18 known phases. Normally ice in nature exists only as Ice Ih, Ice Ic, or a stacking disordered mixture of both. Although many theoretical efforts have been devoted to understanding the thermodynamics of different ice phases at ambient temperature and pressure, there still remains many puzzles. We simulated the reversible transitions between water and different ice phases by performing full atom molecular dynamics simulations. Using the enhanced sampling method MetaITS with the two selected X-ray diffraction peak intensities as collective variables, the ternary phase diagrams of liquid water, ice Ih, ice Ic at multiple were obtained. We also present a simple physical model which successfully explains the thermodynamic stability of ice. Our results agree with experiments and leads to a deeper understanding of the ice nucleation mechanism.

cond-mat.stat-mech↗

arXiv2021

Nonparametric Empirical Bayes Estimation and Testing for Sparse and Heteroscedastic Signals

Large-scale modern data often involves estimation and testing for high-dimensional unknown parameters. It is desirable to identify the sparse signals, ``the needles in the haystack'', with accuracy and false discovery control. However, the unprecedented complexity and heterogeneity in modern data structure require new machine learning tools to effectively exploit commonalities and to robustly adjust for both sparsity and heterogeneity. In addition, estimates for high-dimensional parameters often lack uncertainty quantification. In this paper, we propose a novel Spike-and-Nonparametric mixture prior (SNP) -- a spike to promote the sparsity and a nonparametric structure to capture signals. In contrast to the state-of-the-art methods, the proposed methods solve the estimation and testing problem at once with several merits: 1) an accurate sparsity estimation; 2) point estimates with shrinkage/soft-thresholding property; 3) credible intervals for uncertainty quantification; 4) an optimal multiple testing procedure that controls false discovery rate. Our method exhibits promising empirical performance on both simulated data and a gene expression case study.

cs.LG↗

arXiv2021

Re-Evaluating Strengthened-IV Designs: Asymptotic Efficiency, Bias Formula, and the Validity and Power of Sensitivity Analyses

Instrumental variables (IVs) are extensively used to estimate treatment effects when the treatment and outcome are confounded by unmeasured confounders; however, weak IVs are often encountered in empirical studies and may cause problems. Many studies have considered building a stronger IV from the original, possibly weak, IV in the design stage of a matched study at the cost of not using some of the samples in the analysis. It is widely accepted that strengthening an IV tends to render nonparametric tests more powerful and will increase the power of sensitivity analyses in large samples. In this article, we re-evaluate this conventional wisdom to bring new insights into this topic. We consider matched observational studies from three perspectives. First, we evaluate the trade-off between IV strength and sample size on nonparametric tests assuming the IV is valid and exhibit conditions under which strengthening an IV increases power and conversely conditions under which it decreases power. Second, we derive a necessary condition for a valid sensitivity analysis model with continuous doses. We show that the $Γ$ sensitivity analysis model, which has been previously used to come to the conclusion that strengthening an IV increases the power of sensitivity analyses in large samples, does not apply to the continuous IV setting and thus this previously reached conclusion may be invalid. Third, we quantify the bias of the Wald estimator with a possibly invalid IV under an oracle and leverage it to develop a valid sensitivity analysis framework; under this framework, we show that strengthening an IV may amplify or mitigate the bias of the estimator, and may or may not increase the power of sensitivity analyses. We also discuss how to better adjust for the observed covariates when building an IV in matched studies.

stat.ME↗

arXiv2021

Spatially separated superconductivity and enhanced charge-density-wave ordering in IrTe2 nano-flake

The interplay among various collective electronic states such as superconductivity (SC) and charge density wave (CDW) is of tremendous significance in low-dimensional transition metal dichalcogenides. Thus far, a consensus on the relationship between SC and CDW has not been established in IrTe2, where either competing or collaboration pictures have been suggested in the bulk or thick flakes. Here, we use the state-of-art Au-assisted exfoliation method to overcome the obstacle of interlayer Te-Te bonding, cleaving the IrTe2 down to monolayer for the first time. A striking feature revealed by angle-resolved polarized Raman spectroscopy (ARPRS) and electrical transport measurements is the concurrence of phase separation in one single piece of nano-flake, i.e. the superconducting (P-3m1) and CDW (P-3) area. In the pure SC area, the dimensional fluctuations completely suppress the CDW ordering and induce SC at 3.5 K. Meanwhile, the pure CDW area with much enhanced TCDW at 605 K (compared to TCDW = 280 K in bulk) is always accompanied by a unique wrinkle pattern. Detailed analyses suggest the local strain-induced bond breaking of the Te-Te dimer facilitates the emergence of the CDW order. Our works provide compelling evidence of competition between SC and CDW, highlighting the importance of microstructure in determining the ground states of IrTe2.

cond-mat.supr-con↗

arXiv2021

Competition of superconductivity and charge density wave in selective oxidized CsV3Sb5 thin flakes

The recently discovered layered kagome metals AV$_3$Sb$_5$ (A = K, Rb, and Cs) with vanadium kagome networks provide a novel platform to explore correlated quantum states intertwined with topological band structures. Here we report the prominent effect of hole doping on both superconductivity and charge density wave (CDW) order, achieved by selective oxidation of exfoliated thin flakes. A superconducting dome is revealed as a function of the effective doping content. The superconducting transition temperature ($T_{\mathrm{c}}$) and upper critical field in thin flakes are significantly enhanced compared with the bulk, which is accompanied by the suppression of CDW. Our detailed analyses establish the pivotal role of van Hove singularities (VHSs) in promoting correlated quantum orders in these kagome metals. Our experiment not only demonstrates the intriguing nature of superconducting and CDW orders, but also provides a novel route to tune the carrier concentration through both selective oxidation and electric gating. This establishes AV$_3$Sb$_5$ as a tunable 2D platform for the further exploration of topology and correlation among 3$d$ electrons in kagome lattices.

cond-mat.supr-con↗

arXiv2021

PTR: Prompt Tuning with Rules for Text Classification

Fine-tuned pre-trained language models (PLMs) have achieved awesome performance on almost all NLP tasks. By using additional prompts to fine-tune PLMs, we can further stimulate the rich knowledge distributed in PLMs to better serve downstream tasks. Prompt tuning has achieved promising results on some few-class classification tasks such as sentiment classification and natural language inference. However, manually designing lots of language prompts is cumbersome and fallible. For those auto-generated prompts, it is also expensive and time-consuming to verify their effectiveness in non-few-shot scenarios. Hence, it is still challenging for prompt tuning to address many-class classification tasks. To this end, we propose prompt tuning with rules (PTR) for many-class text classification and apply logic rules to construct prompts with several sub-prompts. In this way, PTR is able to encode prior knowledge of each class into prompt tuning. We conduct experiments on relation classification, a typical and complicated many-class classification task, and the results show that PTR can significantly and consistently outperform existing state-of-the-art baselines. This indicates that PTR is a promising approach to take advantage of both human prior knowledge and PLMs for those complicated classification tasks.

cs.CL↗

arXiv2021

Few-NERD: A Few-Shot Named Entity Recognition Dataset

Recently, considerable literature has grown up around the theme of few-shot named entity recognition (NER), but little published benchmark data specifically focused on the practical and challenging task. Current approaches collect existing supervised NER datasets and re-organize them to the few-shot setting for empirical study. These strategies conventionally aim to recognize coarse-grained entity types with few examples, while in practice, most unseen entity types are fine-grained. In this paper, we present Few-NERD, a large-scale human-annotated few-shot NER dataset with a hierarchy of 8 coarse-grained and 66 fine-grained entity types. Few-NERD consists of 188,238 sentences from Wikipedia, 4,601,160 words are included and each is annotated as context or a part of a two-level entity type. To the best of our knowledge, this is the first few-shot NER dataset and the largest human-crafted NER dataset. We construct benchmark tasks with different emphases to comprehensively assess the generalization capability of models. Extensive empirical results and analysis show that Few-NERD is challenging and the problem requires further research. We make Few-NERD public at https://ningding97.github.io/fewnerd/.

cs.CL↗

arXiv2021

Prompt-Learning for Fine-Grained Entity Typing

As an effective approach to tune pre-trained language models (PLMs) for specific tasks, prompt-learning has recently attracted much attention from researchers. By using \textit{cloze}-style language prompts to stimulate the versatile knowledge of PLMs, prompt-learning can achieve promising results on a series of NLP tasks, such as natural language inference, sentiment classification, and knowledge probing. In this work, we investigate the application of prompt-learning on fine-grained entity typing in fully supervised, few-shot and zero-shot scenarios. We first develop a simple and effective prompt-learning pipeline by constructing entity-oriented verbalizers and templates and conducting masked language modeling. Further, to tackle the zero-shot regime, we propose a self-supervised strategy that carries out distribution-level optimization in prompt-learning to automatically summarize the information of entity types. Extensive experiments on three fine-grained entity typing benchmarks (with up to 86 classes) under fully supervised, few-shot and zero-shot settings show that prompt-learning methods significantly outperform fine-tuning baselines, especially when the training data is insufficient.

cs.CL↗

arXiv2021

Robust low-rank multilinear tensor approximation for a joint estimation of the multilinear rank and the loading matrices

In order to compute the best low-rank tensor approximation using the Multilinear Tensor Decomposition (MTD) model, it is essential to estimate the rank of the underlying multilinear tensor from the noisy observation tensor. In this paper, we propose a Robust MTD (R-MTD) method, which jointly estimates the multilinear rank and the loading matrices. Based on the low-rank property and an over-estimation of the core tensor, this joint estimation problem is solved by promoting (group) sparsity of the over-estimated core tensor. Group sparsity is promoted using mixed-norms. Then we establish a link between the mixed-norms and the nuclear norm, showing that mixed-norms are better candidates for a convex envelope of the rank. After several iterations of the Alternating Direction Method of Multipliers (ADMM), the Minimum Description Length (MDL) criterion computed from the eigenvalues of the unfolding matrices of the estimated core tensor is minimized in order to estimate the multilinear rank. The latter is then used to estimate more accurately the loading matrices. We further develop another R-MTD method, called R-OMTD, by imposing an orthonormality constraint on each loading matrix in order to decrease the computation complexity. A series of simulated noisy tensor and real-world data are used to show the effectiveness of the proposed methods compared with state-of-the-art methods.

cs.CV↗

arXiv2021

Visual Distant Supervision for Scene Graph Generation

Scene graph generation aims to identify objects and their relations in images, providing structured image representations that can facilitate numerous applications in computer vision. However, scene graph models usually require supervised learning on large quantities of labeled data with intensive human annotation. In this work, we propose visual distant supervision, a novel paradigm of visual relation learning, which can train scene graph models without any human-labeled data. The intuition is that by aligning commonsense knowledge bases and images, we can automatically create large-scale labeled data to provide distant supervision for visual relation learning. To alleviate the noise in distantly labeled data, we further propose a framework that iteratively estimates the probabilistic relation labels and eliminates the noisy ones. Comprehensive experimental results show that our distantly supervised model outperforms strong weakly supervised and semi-supervised baselines. By further incorporating human-labeled data in a semi-supervised fashion, our model outperforms state-of-the-art fully supervised models by a large margin (e.g., 8.3 micro- and 7.8 macro-recall@50 improvements for predicate classification in Visual Genome evaluation). We make the data and code for this paper publicly available at https://github.com/thunlp/VisualDS.

cs.CV↗

arXiv2021

OpenCDA:An Open Cooperative Driving Automation Framework Integrated with Co-Simulation

Although Cooperative Driving Automation (CDA) has attracted considerable attention in recent years, there remain numerous open challenges in this field. The gap between existing simulation platforms that mainly concentrate on single-vehicle intelligence and CDA development is one of the critical barriers, as it inhibits researchers from validating and comparing different CDA algorithms conveniently. To this end, we propose OpenCDA, a generalized framework and tool for developing and testing CDA systems. Specifically, OpenCDA is composed of three major components: a co-simulation platform with simulators of different purposes and resolutions, a full-stack cooperative driving system, and a scenario manager. Through the interactions of these three components, our framework offers a straightforward way for researchers to test different CDA algorithms at both levels of traffic and individual autonomy. More importantly, OpenCDA is highly modularized and installed with benchmark algorithms and test cases. Users can conveniently replace any default module with customized algorithms and use other default modules of the CDA platform to perform evaluations of the effectiveness of new functionalities in enhancing the overall CDA performance. An example of platooning implementation is used to illustrate the framework's capability for CDA research. The codes of OpenCDA are available in the https://github.com/ucla-mobility/OpenCDA.

cs.RO↗

arXiv2021

Pre-Trained Models: Past, Present and Future

Large-scale pre-trained models (PTMs) such as BERT and GPT have recently achieved great success and become a milestone in the field of artificial intelligence (AI). Owing to sophisticated pre-training objectives and huge model parameters, large-scale PTMs can effectively capture knowledge from massive labeled and unlabeled data. By storing knowledge into huge parameters and fine-tuning on specific tasks, the rich knowledge implicitly encoded in huge parameters can benefit a variety of downstream tasks, which has been extensively demonstrated via experimental verification and empirical analysis. It is now the consensus of the AI community to adopt PTMs as backbone for downstream tasks rather than learning models from scratch. In this paper, we take a deep look into the history of pre-training, especially its special relation with transfer learning and self-supervised learning, to reveal the crucial position of PTMs in the AI development spectrum. Further, we comprehensively review the latest breakthroughs of PTMs. These breakthroughs are driven by the surge of computational power and the increasing availability of data, towards four important directions: designing effective architectures, utilizing rich contexts, improving computational efficiency, and conducting interpretation and theoretical analysis. Finally, we discuss a series of open problems and research directions of PTMs, and hope our view can inspire and advance the future study of PTMs.

cs.AI↗

arXiv2021

Local Temperature Scaling for Probability Calibration

For semantic segmentation, label probabilities are often uncalibrated as they are typically only the by-product of a segmentation task. Intersection over Union (IoU) and Dice score are often used as criteria for segmentation success, while metrics related to label probabilities are not often explored. However, probability calibration approaches have been studied, which match probability outputs with experimentally observed errors. These approaches mainly focus on classification tasks, but not on semantic segmentation. Thus, we propose a learning-based calibration method that focuses on multi-label semantic segmentation. Specifically, we adopt a convolutional neural network to predict local temperature values for probability calibration. One advantage of our approach is that it does not change prediction accuracy, hence allowing for calibration as a post-processing step. Experiments on the COCO, CamVid, and LPBA40 datasets demonstrate improved calibration performance for a range of different metrics. We also demonstrate the good performance of our method for multi-atlas brain segmentation from magnetic resonance images.

cs.CV↗

arXiv2021

Domain Generalization under Conditional and Label Shifts via Variational Bayesian Inference

In this work, we propose a domain generalization (DG) approach to learn on several labeled source domains and transfer knowledge to a target domain that is inaccessible in training. Considering the inherent conditional and label shifts, we would expect the alignment of $p(x|y)$ and $p(y)$. However, the widely used domain invariant feature learning (IFL) methods relies on aligning the marginal concept shift w.r.t. $p(x)$, which rests on an unrealistic assumption that $p(y)$ is invariant across domains. We thereby propose a novel variational Bayesian inference framework to enforce the conditional distribution alignment w.r.t. $p(x|y)$ via the prior distribution matching in a latent space, which also takes the marginal label shift w.r.t. $p(y)$ into consideration with the posterior alignment. Extensive experiments on various benchmarks demonstrate that our framework is robust to the label shift and the cross-domain accuracy is significantly improved, thereby achieving superior performance over the conventional IFL counterparts.

cs.LG↗

arXiv2021

Bridging the gap between atomically thin semiconductors and metal leads

Electrically interfacing atomically thin transition metal dichalcogenide semiconductors (TMDSCs) with metal leads is challenging because of undesired interface barriers, which have drastically constrained the electrical performance of TMDSC devices for exploring their unconventional physical properties and realizing potential electronic applications. Here we demonstrate a strategy to achieve nearly barrier-free electrical contacts with few-layer TMDSCs by engineering interfacial bonding distortion. The carrier-injection efficiency of such electrical junction is substantially increased with robust ohmic behaviors from room to cryogenic temperatures. The performance enhancements of TMDSC field-effect transistors are well reflected by the ultralow contact resistance (down to 90 Ohm um in MoS2, towards the quantum limit), the ultrahigh field-effect mobility (up to 358,000 cm2V-1s-1 in WSe2) and the prominent transport characteristics at cryogenic temperatures. This method also offers new possibilities of the local manipulation of structures and electronic properties for TMDSC device design.

cond-mat.mtrl-sci↗

arXiv2021

CPM-2: Large-scale Cost-effective Pre-trained Language Models

In recent years, the size of pre-trained language models (PLMs) has grown by leaps and bounds. However, efficiency issues of these large-scale PLMs limit their utilization in real-world scenarios. We present a suite of cost-effective techniques for the use of PLMs to deal with the efficiency issues of pre-training, fine-tuning, and inference. (1) We introduce knowledge inheritance to accelerate the pre-training process by exploiting existing PLMs instead of training models from scratch. (2) We explore the best practice of prompt tuning with large-scale PLMs. Compared with conventional fine-tuning, prompt tuning significantly reduces the number of task-specific parameters. (3) We implement a new inference toolkit, namely InfMoE, for using large-scale PLMs with limited computational resources. Based on our cost-effective pipeline, we pre-train two models: an encoder-decoder bilingual model with 11 billion parameters (CPM-2) and its corresponding MoE version with 198 billion parameters. In our experiments, we compare CPM-2 with mT5 on downstream tasks. Experimental results show that CPM-2 has excellent general language intelligence. Moreover, we validate the efficiency of InfMoE when conducting inference of large-scale models having tens of billions of parameters on a single GPU. All source code and model parameters are available at https://github.com/TsinghuaAI/CPM.

cs.CL↗