arXiv subjects

Xu Han

Publications and source records attributed to Xu Han.

At least 253 records · Page 14Linked to original sources

arXiv2022

Enhanced sampling reveals the main pathway of an organic cascade reaction

Normal molecular dynamics simulations are usually unable to simulate chemical reactions due to the low probability of forming the transition state. Therefore, enhanced sampling methods are implemented to accelerate the occurrence of chemical reactions. In this investigation, we present an application of metadynamics in simulating an organic multi-step cascade reaction. The analysis of the reaction trajectory reveals the barrier heights of both forward and reverse reactions. We also present a discussion of the advantages and disadvantages of generating the reactive pathway using molecular dynamics and the intrinsic reaction coordinate (IRC) algorithm.

physics.chem-ph↗

arXiv2022

Improving End-to-End Text Image Translation From the Auxiliary Text Translation Task

End-to-end text image translation (TIT), which aims at translating the source language embedded in images to the target language, has attracted intensive attention in recent research. However, data sparsity limits the performance of end-to-end text image translation. Multi-task learning is a non-trivial way to alleviate this problem via exploring knowledge from complementary related tasks. In this paper, we propose a novel text translation enhanced text image translation, which trains the end-to-end model with text translation as an auxiliary task. By sharing model parameters and multi-task training, our model is able to take full advantage of easily-available large-scale text parallel corpus. Extensive experimental results show our proposed method outperforms existing end-to-end methods, and the joint multi-task learning with both text translation and recognition tasks achieves better results, proving translation and recognition auxiliary tasks are complementary.

cs.CL↗

arXiv2022

GACT: Activation Compressed Training for Generic Network Architectures

Training large neural network (NN) models requires extensive memory resources, and Activation Compressed Training (ACT) is a promising approach to reduce training memory footprint. This paper presents GACT, an ACT framework to support a broad range of machine learning tasks for generic NN architectures with limited domain knowledge. By analyzing a linearized version of ACT's approximate gradient, we prove the convergence of GACT without prior knowledge on operator type or model architecture. To make training stable, we propose an algorithm that decides the compression ratio for each tensor by estimating its impact on the gradient at run time. We implement GACT as a PyTorch library that readily applies to any NN architecture. GACT reduces the activation memory for convolutional NNs, transformers, and graph NNs by up to 8.1x, enabling training with a 4.2x to 24.7x larger batch size, with negligible accuracy loss. We implement GACT as a PyTorch library at https://github.com/LiuXiaoxuanPKU/GACT-ICML.

cs.LG↗

arXiv2022

Meta Reinforcement Learning with Successor Feature Based Context

Most reinforcement learning (RL) methods only focus on learning a single task from scratch and are not able to use prior knowledge to learn other tasks more effectively. Context-based meta RL techniques are recently proposed as a possible solution to tackle this. However, they are usually less efficient than conventional RL and may require many trial-and-errors during training. To address this, we propose a novel meta-RL approach that achieves competitive performance comparing to existing meta-RL algorithms, while requires significantly fewer environmental interactions. By combining context variables with the idea of decomposing reward in successor feature framework, our method does not only learn high-quality policies for multiple tasks simultaneously but also can quickly adapt to new tasks with a small amount of training. Compared with state-of-the-art meta-RL baselines, we empirically show the effectiveness and data efficiency of our method on several continuous control tasks.

cs.LG↗

arXiv2022

Recirculating Light Phase Modulator

High efficiency and a compact footprint are desired properties for electro-optic modulators. In this paper, we propose, theoretically investigate and experimentally demonstrate a recirculating phase modulator, which increases the modulation efficiency by modulating the optical field several times in a non-resonant waveguide structure. The 'recycling' of light is achieved by looping the optical path that exits the phase modulator back and coupling it to a higher order waveguide mode, which then repeats its passage through the phase modulator. By looping the light back twice, we were able to demonstrate a recirculating phase modulator that requires nine times lower power to generate the same modulation index of a single pass phase modulator. This approach of modulation efficiency enhancement is promising for the design of advanced tunable electro optical frequency comb generators and other electro-optical devices with defined operational frequency bandwidths.

physics.optics↗

arXiv2022

Superconducting cavity piezo-electromechanics: the realization of an acoustic frequency comb at microwave frequencies

We present a nonlinear multimode superconducting electroacoustic system, where the interplay between superconducting kinetic inductance and piezoelectric strong coupling establishes an effective Kerr nonlinearity among multiple acoustic modes at 10 GHz that could hardly be achieved via intrinsic mechanical nonlinearity. By exciting this multimode Kerr system with a single microwave tone, we further demonstrate a coherent electroacoustic frequency comb and provide theoretical understanding of multimode nonlinear interaction in the superstrong coupling limit. This nonlinear superconducting electroacoustic system sheds light on the active control of multimode resonator systems and offers an enabling platform for the dynamic study of microcombs at microwave frequencies.

physics.app-ph↗

arXiv2022

One-step exfoliation method for plasmonic activation of large-area 2D crystals

Advanced exfoliation techniques are crucial for exploring the intrinsic properties and applications of 2D materials. Though the recently discovered Au-enhanced exfoliation technique provides an effective strategy for preparation of large-scale 2D crystals, the high cost of gold hinders this method from being widely adopted in industrial applications. In addition, direct Au contact could significantly quench photoluminescence (PL) emission in 2D semiconductors. It is therefore crucial to find alternative metals that can replace gold to achieve efficient exfoliation of 2D materials. Here, we present a one-step Ag-assisted method that can efficiently exfoliate many large-area 2D monolayers, where the yield ratio is comparable to Au-enhanced exfoliation method. Differing from Au film, however, the surface roughness of as-prepared Ag films on SiO2/Si substrate is much higher, which facilitates the generation of surface plasmons resulting from the nanostructures formed on the rough Ag surface. More interestingly, the strong coupling between 2D semiconductor crystals (e.g. MoS2, MoSe2) and Ag film leads to a unique PL enhancement that has not been observed in other mechanical exfoliation techniques, which can be mainly attributed to enhanced light-matter interaction as a result of extended propagation of surface plasmonic polariton (SPP). Our work provides a lower-cost and universal Ag-assisted exfoliation method, while at the same offering enhanced SPP-matter interactions.

cond-mat.mtrl-sci↗

arXiv2022

OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication

Employing Vehicle-to-Vehicle communication to enhance perception performance in self-driving technology has attracted considerable attention recently; however, the absence of a suitable open dataset for benchmarking algorithms has made it difficult to develop and assess cooperative perception technologies. To this end, we present the first large-scale open simulated dataset for Vehicle-to-Vehicle perception. It contains over 70 interesting scenes, 11,464 frames, and 232,913 annotated 3D vehicle bounding boxes, collected from 8 towns in CARLA and a digital town of Culver City, Los Angeles. We then construct a comprehensive benchmark with a total of 16 implemented models to evaluate several information fusion strategies~(i.e. early, late, and intermediate fusion) with state-of-the-art LiDAR detection algorithms. Moreover, we propose a new Attentive Intermediate Fusion pipeline to aggregate information from multiple connected vehicles. Our experiments show that the proposed pipeline can be easily integrated with existing 3D LiDAR detectors and achieve outstanding performance even with large compression rates. To encourage more researchers to investigate Vehicle-to-Vehicle perception, we will release the dataset, benchmark methods, and all related codes in https://mobility-lab.seas.ucla.edu/opv2v/.

cs.CV↗

arXiv2022

Exfoliation of 2D van der Waals crystals in ultrahigh vacuum for interface engineering

Two-dimensional (2D) materials and their heterostructures have been intensively studied in recent years due to their potential applications in electronic, optoelectronic, and spintronic devices. Nonetheless, the realization of 2D heterostructures with atomically flat and clean interfaces remains challenging, especially for air-sensitive materials, which hinders the in-depth investigation of interface-induced phenomena and the fabrication of high-quality devices. Here, we circumvented this challenge by exfoliating 2D materials in an ultrahigh vacuum. Remarkably, ultraflat and clean substrate surfaces can assist the exfoliation of 2D materials, regardless of the substrate and 2D material, thus providing a universal method for the preparation of heterostructures with ideal interfaces. In addition, we studied the properties of two prototypical systems that cannot be achieved previously, including the electronic structure of monolayer phospherene and optical responses of transition metal dichalcogenides on different metal substrates. Our work paves the way to engineer rich interface-induced phenomena, such as proximity effects and moiré superlattices.

cond-mat.mtrl-sci↗

arXiv2022

Predicting Physics in Mesh-reduced Space with Temporal Attention

Graph-based next-step prediction models have recently been very successful in modeling complex high-dimensional physical systems on irregular meshes. However, due to their short temporal attention span, these models suffer from error accumulation and drift. In this paper, we propose a new method that captures long-term dependencies through a transformer-style temporal attention model. We introduce an encoder-decoder structure to summarize features and create a compact mesh representation of the system state, to allow the temporal model to operate on a low-dimensional mesh representations in a memory efficient manner. Our method outperforms a competitive GNN baseline on several complex fluid dynamics prediction tasks, from sonic shocks to vascular flow. We demonstrate stable rollouts without the need for training noise and show perfectly phase-stable predictions even for very long sequences. More broadly, we believe our approach paves the way to bringing the benefits of attention-based sequence models to solving high-dimensional complex physics tasks.

cs.LG↗

arXiv2022

Knowledge Inheritance for Pre-trained Language Models

Recent explorations of large-scale pre-trained language models (PLMs) have revealed the power of PLMs with huge amounts of parameters, setting off a wave of training ever-larger PLMs. However, it requires tremendous computational resources to train a large-scale PLM, which may be practically unaffordable. In addition, existing large-scale PLMs are mainly trained from scratch individually, ignoring that many well-trained PLMs are available. To this end, we explore the question how could existing PLMs benefit training large-scale PLMs in future. Specifically, we introduce a pre-training framework named "knowledge inheritance" (KI) and explore how could knowledge distillation serve as auxiliary supervision during pre-training to efficiently learn larger PLMs. Experimental results demonstrate the superiority of KI in training efficiency. We also conduct empirical analyses to explore the effects of teacher PLMs' pre-training settings, including model architecture, pre-training data, etc. Finally, we show that KI could be applied to domain adaptation and knowledge transfer.

cs.CL↗

arXiv2022

Sampling-based Fast Gradient Rescaling Method for Highly Transferable Adversarial Attacks

Deep neural networks have shown to be very vulnerable to adversarial examples crafted by adding human-imperceptible perturbations to benign inputs. After achieving impressive attack success rates in the white-box setting, more focus is shifted to black-box attacks. In either case, the common gradient-based approaches generally use the $sign$ function to generate perturbations at the end of the process. However, only a few works pay attention to the limitation of the $sign$ function. Deviation between the original gradient and the generated noises may lead to inaccurate gradient update estimation and suboptimal solutions for adversarial transferability, which is crucial for black-box attacks. To address this issue, we propose a Sampling-based Fast Gradient Rescaling Method (S-FGRM) to improve the transferability of the crafted adversarial examples. Specifically, we use data rescaling to substitute the inefficient $sign$ function in gradient-based attacks without extra computational cost. We also propose a Depth First Sampling method to eliminate the fluctuation of rescaling and stabilize the gradient update. Our method can be used in any gradient-based optimizations and is extensible to be integrated with various input transformation or ensemble methods for further improving the adversarial transferability. Extensive experiments on the standard ImageNet dataset show that our S-FGRM could significantly boost the transferability of gradient-based attacks and outperform the state-of-the-art baselines.

cs.CV↗

arXiv2022

A Roadmap for Big Model

With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm. Researchers have achieved various outcomes in the construction of BMs and the BM application in many fields. At present, there is a lack of research work that sorts out the overall progress of BMs and guides the follow-up research. In this paper, we cover not only the BM technologies themselves but also the prerequisites for BM training and applications with BMs, dividing the BM review into four parts: Resource, Models, Key Technologies and Application. We introduce 16 specific BM-related topics in those four parts, they are Data, Knowledge, Computing System, Parallel Training System, Language Model, Vision Model, Multi-modal Model, Theory&Interpretability, Commonsense Reasoning, Reliability&Security, Governance, Evaluation, Machine Translation, Text Generation, Dialogue and Protein Research. In each topic, we summarize clearly the current studies and propose some future research directions. At the end of this paper, we conclude the further development of BMs in a more general view.

cs.LG↗

arXiv2022

Atomic-Scale Visualization of Chiral Charge Density Wave States and Their Reversible Transition

Chirality is essential for various amazing phenomena in life and matter. However,chirality and its switching in electronic superlattices, such as charge density wave(CDW) arrays, remain elusive. In this study, we characterize the chirality transition with atom-resolution imaging in a single-layer NbSe2 CDW pattern by technique of scanning tunneling microscopy. The atomic lattice of the CDW array is found continuous and intact although its chirality is switched. Several intermediate states are tracked by time-resolved imaging, revealing the fast and dynamic chirality transition. Importantly, the switching is reversibly realized with an external electric-field. Our findings unveil the delicate transition process of chiral CDW array in a 2D crystal down to the atomic scale and may be applicable for future nanoscale devices.

cond-mat.mtrl-sci↗

arXiv2022

Fully Hyperbolic Neural Networks

Hyperbolic neural networks have shown great potential for modeling complex data. However, existing hyperbolic networks are not completely hyperbolic, as they encode features in a hyperbolic space yet formalize most of their operations in the tangent space (a Euclidean subspace) at the origin of the hyperbolic space. This hybrid method greatly limits the modeling ability of networks. In this paper, we propose a fully hyperbolic framework to build hyperbolic networks based on the Lorentz model by adapting the Lorentz transformations (including boost and rotation) to formalize essential operations of neural networks. Moreover, we also prove that linear transformation in tangent spaces used by existing hyperbolic networks is a relaxation of the Lorentz rotation and does not include the boost, implicitly limiting the capabilities of existing hyperbolic networks. The experimental results on four NLP tasks show that our method has better performance for building both shallow and deep networks. Our code will be released to facilitate follow-up research.

cs.CL↗

arXiv2022

PPT: Pre-trained Prompt Tuning for Few-shot Learning

Prompts for pre-trained language models (PLMs) have shown remarkable performance by bridging the gap between pre-training tasks and various downstream tasks. Among these methods, prompt tuning, which freezes PLMs and only tunes soft prompts, provides an efficient and effective solution for adapting large-scale PLMs to downstream tasks. However, prompt tuning is yet to be fully explored. In our pilot experiments, we find that prompt tuning performs comparably with conventional full-model fine-tuning when downstream data are sufficient, whereas it performs much worse under few-shot learning settings, which may hinder the application of prompt tuning in practice. We attribute this low performance to the manner of initializing soft prompts. Therefore, in this work, we propose to pre-train prompts by adding soft prompts into the pre-training stage to obtain a better initialization. We name this Pre-trained Prompt Tuning framework "PPT". To ensure the generalization of PPT, we formulate similar classification tasks into a unified task form and pre-train soft prompts for this unified task. Extensive experiments show that tuning pre-trained prompts for downstream tasks can reach or even outperform full-model fine-tuning under both full-data and few-shot settings. Our approach is effective and efficient for using large-scale PLMs in practice.

cs.CL↗

arXiv2022

Single electrons on solid neon as a solid-state qubit platform

Progress toward the realization of quantum computers requires persistent advances in their constituent building blocks - qubits. Novel qubit platforms that simultaneously embody long coherence, fast operation, and large scalability offer compelling advantages in the construction of quantum computers and many other quantum information systems. Electrons, ubiquitous elementary particles of nonzero charge, spin, and mass, have commonly been perceived as paradigmatic local quantum information carriers. Despite superior controllability and configurability, their practical performance as qubits via either motional or spin states depends critically on their material environment. Here we report our experimental realization of a new qubit platform based upon isolated single electrons trapped on an ultraclean solid neon surface in vacuum. By integrating an electron trap in a circuit quantum electrodynamics architecture, we achieve strong coupling between the motional states of a single electron and a single microwave photon in an on-chip superconducting resonator. Qubit gate operations and dispersive readout are implemented to measure the energy relaxation time $T_1$ of $15~μ$s and phase coherence time $T_2$ over $200~$ns. These results indicate that the electron-on-solid-neon qubit already performs near the state of the art as a charge qubit.

quant-ph↗

arXiv2022

Skilled Mutual Fund Selection: False Discovery Control under Dependence

Selecting skilled mutual funds through the multiple testing framework has received increasing attention from finance researchers and statisticians. The intercept $α$ of Carhart four-factor model is commonly used to measure the true performance of mutual funds, and positive $α$'s are considered as skilled. We observe that the standardized OLS estimates of $α$'s across the funds possess strong dependence and nonnormality structures, indicating that the conventional multiple testing methods are inadequate for selecting the skilled funds. We start from a decision theoretic perspective, and propose an optimal testing procedure to minimize a combination of false discovery rate and false non-discovery rate. Our proposed testing procedure is constructed based on the probability of each fund not being skilled conditional on the information across all of the funds in our study. To model the distribution of the information used for the testing procedure, we consider a mixture model under dependence and propose a new method called "approximate empirical Bayes" to fit the parameters. Empirical studies show that our selected skilled funds have superior long-term and short-term performance, e.g., our selection strongly outperforms the S\&P 500 index during the same period.

stat.ME↗