arXiv subjects

Liang Xu

Publications and source records attributed to Liang Xu.

At least 127 records · Page 7Linked to original sources

arXiv2021

Yuan 1.0: Large-Scale Pre-trained Language Model in Zero-Shot and Few-Shot Learning

Recent work like GPT-3 has demonstrated excellent performance of Zero-Shot and Few-Shot learning on many natural language processing (NLP) tasks by scaling up model size, dataset size and the amount of computation. However, training a model like GPT-3 requires huge amount of computational resources which makes it challengeable to researchers. In this work, we propose a method that incorporates large-scale distributed training performance into model architecture design. With this method, Yuan 1.0, the current largest singleton language model with 245B parameters, achieves excellent performance on thousands GPUs during training, and the state-of-the-art results on NLP tasks. A data processing method is designed to efficiently filter massive amount of raw data. The current largest high-quality Chinese corpus with 5TB high quality texts is built based on this method. In addition, a calibration and label expansion method is proposed to improve the Zero-Shot and Few-Shot performance, and steady improvement is observed on the accuracy of various tasks. Yuan 1.0 presents strong capacity of natural language generation, and the generated articles are difficult to distinguish from the human-written ones.

cs.CL↗

arXiv2021

FewCLUE: A Chinese Few-shot Learning Evaluation Benchmark

Pretrained Language Models (PLMs) have achieved tremendous success in natural language understanding tasks. While different learning schemes -- fine-tuning, zero-shot, and few-shot learning -- have been widely explored and compared for languages such as English, there is comparatively little work in Chinese to fairly and comprehensively evaluate and compare these methods and thus hinders cumulative progress. In this paper, we introduce the Chinese Few-shot Learning Evaluation Benchmark (FewCLUE), the first comprehensive few-shot evaluation benchmark in Chinese. It includes nine tasks, ranging from single-sentence and sentence-pair classification tasks to machine reading comprehension tasks. We systematically evaluate five state-of-the-art (SOTA) few-shot learning methods (including PET, ADAPET, LM-BFF, P-tuning and EFL), and compare their performance with fine-tuning and zero-shot learning schemes on the newly constructed FewCLUE benchmark. Experimental results reveal that: 1) The effect of different few-shot learning methods is sensitive to the pre-trained model to which the methods are applied; 2) PET and P-tuning achieve the best overall performance with RoBERTa and ERNIE respectively. Our benchmark is used in the few-shot learning contest of NLPCC 2021. In addition, we provide a user-friendly toolkit, as well as an online leaderboard to help facilitate further progress on Chinese few-shot learning. We provide a baseline performance on different learning methods, a reference for future research.

cs.CL↗

arXiv2021

An Alignment-Agnostic Model for Chinese Text Error Correction

This paper investigates how to correct Chinese text errors with types of mistaken, missing and redundant characters, which is common for Chinese native speakers. Most existing models based on detect-correct framework can correct mistaken characters errors, but they cannot deal with missing or redundant characters. The reason is that lengths of sentences before and after correction are not the same, leading to the inconsistence between model inputs and outputs. Although the Seq2Seq-based or sequence tagging methods provide solutions to the problem and achieved relatively good results on English context, but they do not perform well in Chinese context according to our experimental results. In our work, we propose a novel detect-correct framework which is alignment-agnostic, meaning that it can handle both text aligned and non-aligned occasions, and it can also serve as a cold start model when there are no annotated data provided. Experimental results on three datasets demonstrate that our method is effective and achieves the best performance among existing published models.

cs.CL↗

arXiv2021

Measurement of undulator section wakefield at the SXFEL test facility

In free electron laser facilities, almost every kind of device will generate wakefield when an electron beam passes through it. Most of the wakefields are undesired and have a negative effect on the electron beam, which means a decrease of FEL performance. As for the SXFEL test facility, the sophisticated layout and the cumulative effect of such a long undulator section lead to an obvious wakefield, which is strong enough that can not be ignored. Based on two deflecting cavities at the entrance and the exit of the undulator section with corresponding profile monitors, we measured the wakefield of the undulator section. In this paper, we give the theoretical and simulation results of resistive wall wakefields which agree well with each other. In addition, the experimental and the simulation results of the overall undulator wakefield are given showing small difference. In order to explore the impact of this wakefield on FEL lasing, we give the simulation results of FEL with and without wakefield for comparison. There is almost no impact on 44 nm FEL in stage-1 of cascaded EEHG-HGHG mode, while the impact on 8.8 nm FEL in stage-2 becomes critical decreasing the pulse energy and peak power by 42% and 27% and broadening the bandwidth.

physics.acc-ph↗

arXiv2021

Numerical Study on Beam-based Alignment of SXFEL Undulator Lattice

The undulator line of the Shanghai soft X-ray Free-electron Laser facility (SXFEL) has very tight tolerances on the straightness of the electron beam trajectory. However, the beam trajectory cannot meet the lasing requirements due to the influence of beam position, launch angle and quadrupole offsets. Traditional mechanical alignment can only control the rms of offsets to about 100 $μ$m, which is far from reaching the requirement. Further orbit correction can be achieved by beam-based alignment (BBA) method based on electron energy variations. K modulation is used to determine whether the beam passes through the quadrupole magnetic center, and the Dispersion-Free Steering (DFS) method is used to calculate the offsets of quadrupole and BPM. In this paper, a detailed result of simulation is presented which demonstrates that the beam trajectory with rms and standard deviation ($σ$) less than 10 $μ$m can be obtained.

physics.acc-ph↗

arXiv2021

Quantum-limited Localisation and Resolution in Three Dimensions

As a method to extract information from optical system, imaging can be viewed as a parameter estimation problem. The fundamental precision in locating one emitter or estimating the separation between two incoherent emitters is bounded below by the multiparameter quantum Cramer-Rao bound (QCRB).Multiparameter QCRB gives an intrinsic bound in parameter estimation. We determine the ultimate potential of quantum-limited imaging for improving the resolution of a far-field, diffraction-limited within the paraxial approximation. We show that the quantum Fisher information matrix (QFIm) about one emitter's position is independent on the true value of it. We calculate the QFIm of two unequal-brightness emitters' relative positions and intensities, the results show that only when the relative intensity and centroids of two point sources including longitudinal and transverse direction are known exactly, the separation in different directions can be estimated simultaneously with finite precision. Our results give the upper bounds on certain far-field imaging technology and will find wide applications from microscopy to astrometry.

quant-ph↗

arXiv2021

Experimental Quantum Target Detection Approaching the Fundamental Helstrom Limit

Quantum target detection is an emerging application that utilizes entanglement to enhance the sensing of the presence of an object. Although several experimental demonstrations for certain situations have been reported recently, the single-shot detection limit imposed by the Helstrom limit has not been reached because of the unknown optimum measurements. Here we report an experimental demonstration of quantum target detection, also known as quantum illumination, in the single-photon limit. In our experiment, one photon of the maximally entangled photon pair is employed as the probe signal and the corresponding optimum measurement is implemented at the receiver. We explore the detection problem in different regions of the parameter space and verify that the quantum advantage exists even in a forbidden region of the conventional illumination, where all classical schemes become useless. Our results indicate that quantum illumination breaks the classical limit for up to 40%, while approaching the quantum limit imposed by the Helstrom limit. These results not only demonstrate the advantage of quantum illumination, but also manifest its valuable potential of target detection.

quant-ph↗

arXiv2021

Non-conservative Design of Robust Tracking Controllers Based on Input-output Data

This paper studies worst-case robust optimal tracking using noisy input-output data. We utilize behavioral system theory to represent system trajectories, while avoiding explicit system identification. We assume that the recent output data used in the data-dependent representation are noisy and we provide a non-conservative design procedure for robust control based on optimization with a linear cost and LMI constraints. Our methods rely on the parameterization of noise sequences compatible with the data-dependent system representation and on a suitable reformulation of the performance specification, which further enable the application of the S-lemma to derive an LMI optimization problem. The performance of the new controller is discussed through simulations.

math.OC↗

arXiv2021

Reverberation Mapping Measurements of Black Hole Masses and Broad-Line Region Kinematics in Mrk 817 and NGC 7469

We present the results from a spectroscopic monitoring campaign to obtain reverberation-mapping measurements and investigate the broad-line region kinematics for active galactic nuclei (AGN) of Mrk~817 and NGC~7469. This campaign was undertaken with the Lijiang 2.4-meter telescope, the median spectroscopic sampling is 2.0 days for Mrk~817 and 1.0 days for NGC~7469. We detect time lags of the broad emission lines including H$β$, H$γ$, He~{\sc ii} and He~{\sc i} for both AGNs, and including Fe~{\sc ii} for Mrk~817 with respect to the varying AGN continuum at 5100~Å. Investigating the relationship between line widths and time lags of the broad emission lines, we find that the BLR dynamics of Mrk~817 and NGC~7469 are consistent with the virial prediction. We estimate the masses of central supermassive black hole (SMBH) and the accretion rates of both AGNs. Using the data of this campaign, we construct the velocity-resolved lag profiles of the broad H$γ$, H$β$, and He~{\sc i} lines for Mrk~817, which show almost the same kinematic signatures that the time lags in the red wing are slightly larger than the time lags in the blue wing. For NGC~7469, we only clearly construct the velocity-resolved lag profiles of the broad H$γ$ and H$β$, which show very similar kinematic signatures to the BLR of Mrk~817. These signatures indicate that the BLR of Keplerian motion in both AGNs seemingly has outflowing components during the monitoring period. We discuss the kinematics of the BLR and the measurements including SMBH mass and accretion rates.

astro-ph.GA↗

arXiv2021

On Consensusability of Linear Interconnected Multi-Agent Systems and Simultaneous Stabilization

Consensusability of multi-agent systems (MASs) certifies the existence of a distributed controller capable of driving the states of each subsystem to a consensus value. We study the consensusability of linear interconnected MASs (LIMASs) where, as in several real-world applications, subsystems are physically coupled. We show that consensusability is related to the simultaneous stabilizability of multiple LTI systems, and present a novel sufficient condition in form of a linear program for verifying this property. We also derive several necessary and sufficient consensusability conditions for LIMASs in terms of parameters of the subsystem matrices and the eigenvalues of the physical and communication graph Laplacians. The results show that weak physical couplings among subsystems and densely-connected physical and communication graphs are favorable for consensusability. Finally, we validate our results through simulations of networks of supercapacitors and DC microgrids.

eess.SY↗

arXiv2021

A unified framework for Hamiltonian deep neural networks

Training deep neural networks (DNNs) can be difficult due to the occurrence of vanishing/exploding gradients during weight optimization. To avoid this problem, we propose a class of DNNs stemming from the time discretization of Hamiltonian systems. The time-invariant version of the corresponding Hamiltonian models enjoys marginal stability, a property that, as shown in previous works and for specific DNNs architectures, can mitigate convergence to zero or divergence of gradients. In the present paper, we formally study this feature by deriving and analysing the backward gradient dynamics in continuous time. The proposed Hamiltonian framework, besides encompassing existing networks inspired by marginally stable ODEs, allows one to derive new and more expressive architectures. The good performance of the novel DNNs is demonstrated on benchmark classification problems, including digit recognition using the MNIST dataset.

cs.LG↗

arXiv2021

Improving the Precision of Optical Metrology by Detecting Fewer Photons

In optical metrological protocols to measure physical quantities, it is, in principle, always beneficial to increase photon number to improve measurement precision. However, practical constraints prevent arbitrary increase of n due to the imperfections of a practical detector, especially when the detector response is dominated by saturation effect. In this work, we show that a modified weak measurement protocol, namely, biased weak measurement significantly improves the precision of optical metrology in the presence of saturation effect. This method detects an ultra-small fraction of photons while maintains considerable amount of metrological information. The biased pre-coupling leads to an additional reduction of photons in the post-selection and generates an extinction point in the spectrum distribution, which is extremely sensitive to the estimated parameter and difficult to be saturated. Therefore, the Fisher information can be persistently enhanced by increasing the photon number. In our magnetic-sensing experiment, biased weak measurement achieves precision approximately one order of magnitude better than those of previously used methods. The proposed method can be applied in various optical measurement schemes to circumvent detector saturation effect with low-cost apparatuses.

quant-ph↗

arXiv2021

A correction method for the telluric absorptions and application to Lijiang Observatory

Observing a telluric standard star for correcting the telluric absorption lines of spectrum will take a significant amount of precious telescope time, especially in the long-term spectral monitoring project. Beyond that, it's difficult to select a suitable telluric standard star near in both time and airmass to the scientific object. In this paper, we present a method of correcting the telluric absorption lines by combining the advantages of long slit spectroscopy. By rotating the slit, we observed the scientific object and a nearby comparison star in one exposure, so that the spectra of both objects should have the same telluric transmission spectrum. The telluric transmission spectrum was constructed by dividing the observed spectrum of comparison star by its stellar template, and was used to correct the telluric absorption lines of the scientific object. Using the long slit spectrograph of Lijiang 2.4-meter telescope, we designed a long-term spectroscopic observation strategy, and finished a four-year spectroscopic monitoring for a pair of objects (an active galactic nuclei and an non-varying comparison star). We applied this method to correct the telluric absorption lines of the long-term monitored spectra by Lijiang 2.4-meter telescope, and investigated the variation of the telluric absorptions at Lijiang Observatory. We found that the telluric absorption transparency is mainly modulated by the seasonal variability of the relative humidity, airmass and seeing. Using the scatter of the [O~III] $λ$5007 fluxes emitted from the narrow-line region of active galactic nuclei as an indicator, we found that the correction accuracy of the telluric absorption lines is 1%.

astro-ph.IM↗

arXiv2021

Direct evidence of the gradient drift instability being the origin of a rotating spoke in a crossed field plasma

A plasma rotating spoke in a crossed field discharge is studied using 2D radial-azimuthal fully kinetic Particle-In-Cell Monte Carlo Collision (PIC/MCC) simulations. The kinetic model reveals the whole perturbation spectrum of the gradient drift instability in the linear stage: Simon-Hoh, lower-hybrid and ion sound modes, providing direct evidence of the spoke of the gradient drift instability nature. The two-fluid dispersion relation of the gradient drift instability was utilized to analyze the linear development of instabilities in the simulations. The charge separation effect was incorporated in the fluid linear theory and a super-resolution signal processing method (multiple signal classification) was applied to obtain the numerical frequency spectrum. The simulated spectrum and growth rate show excellent agreement with the theoretical dispersion relation (real frequency and imaginary frequency) over investigated cases. The most linearly unstable mode was found to be the lower hybrid instability and the mode transition into the m=1 macroscopic rotating structure after saturation of the linear phase is accompanied by an inverse energy cascade. In the nonlinear stage, the pronounced spoke phenomena can occur when the heating of $\mathbf{E_θ\times B}$ electron flow channeled in the spoke front passage suffices to provide the enhanced ionization.

physics.plasm-ph↗

arXiv2021

Transferable Interactiveness Knowledge for Human-Object Interaction Detection

Human-Object Interaction (HOI) detection is an important problem to understand how humans interact with objects. In this paper, we explore interactiveness knowledge which indicates whether a human and an object interact with each other or not. We found that interactiveness knowledge can be learned across HOI datasets and bridge the gap between diverse HOI category settings. Our core idea is to exploit an interactiveness network to learn the general interactiveness knowledge from multiple HOI datasets and perform Non-Interaction Suppression (NIS) before HOI classification in inference. On account of the generalization ability of interactiveness, interactiveness network is a transferable knowledge learner and can be cooperated with any HOI detection models to achieve desirable results. We utilize the human instance and body part features together to learn the interactiveness in hierarchical paradigm, i.e., instance-level and body part-level interactivenesses. Thereafter, a consistency task is proposed to guide the learning and extract deeper interactive visual clues. We extensively evaluate the proposed method on HICO-DET, V-COCO, and a newly constructed PaStaNet-HOI dataset. With the learned interactiveness, our method outperforms state-of-the-art HOI detection methods, verifying its efficacy and flexibility. Code is available at https://github.com/DirtyHarryLYL/Transferable-Interactiveness-Network.

cs.CV↗

arXiv2021

A Data-Driven Convex Programming Approach to Worst-Case Robust Tracking Controller Design

This paper studies finite-horizon robust tracking control for discrete-time linear systems, based on input-output data. We leverage behavioral theory to represent system trajectories through a set of noiseless historical data, instead of using an explicit system model. By assuming that recent output data available to the controller are affected by noise terms verifying a quadratic bound, we formulate an optimization problem with a linear cost and LMI constraints for solving the robust tracking problem without any approximations. Our approach hinges on a parameterization of noise trajectories compatible with the data-dependent system representation and on a reformulation of the tracking cost, which enables the application of the S-lemma. In addition, we propose a method for reducing the computational complexity and demonstrate that the size of the resulting LMIs does not scale with the number of historical data. Finally, we show that the proposed formulation can easily incorporate actuator disturbances as well as constraints on inputs and outputs. The performance of the new controllers is discussed through simulations.

math.OC↗

arXiv2021

Pushing the Envelope of Thin Crack Detection

In this study, we consider the problem of detecting cracks from the image of a concrete surface for automated inspection of infrastructure, such as bridges. Its overall accuracy is determined by how accurately thin cracks with sub-pixel widths can be detected. Our interest is in making it possible to detect cracks close to the limit of thinness if it can be defined. Toward this end, we first propose a method for training a CNN to make it detect cracks more accurately than humans while training them on human-annotated labels. To achieve this seemingly impossible goal, we intentionally lower the spatial resolution of input images while maintaining that of their labels when training a CNN. This makes it possible to annotate cracks that are too thin for humans to detect, which we call super-human labels. We experimentally show that this makes it possible to detect cracks from an image of one-third the resolution of images used for annotation with about the same accuracy. We additionally propose three methods for further improving the detection accuracy of thin cracks: i) P-pooling to maintain small image structures during downsampling operations; ii) Removal of short-segment cracks in a post-processing step utilizing a prior of crack shapes learned using the VAE-GAN framework; iii) Modeling uncertainty of the prediction to better handle hard labels beyond the limit of CNNs' detection ability, which technically work as noisy labels. We experimentally examine the effectiveness of these methods.

cs.CV↗

arXiv2021

NVAE-GAN Based Approach for Unsupervised Time Series Anomaly Detection

In recent studies, Lots of work has been done to solve time series anomaly detection by applying Variational Auto-Encoders (VAEs). Time series anomaly detection is a very common but challenging task in many industries, which plays an important role in network monitoring, facility maintenance, information security, and so on. However, it is very difficult to detect anomalies in time series with high accuracy, due to noisy data collected from real world, and complicated abnormal patterns. From recent studies, we are inspired by Nouveau VAE (NVAE) and propose our anomaly detection model: Time series to Image VAE (T2IVAE), an unsupervised model based on NVAE for univariate series, transforming 1D time series to 2D image as input, and adopting the reconstruction error to detect anomalies. Besides, we also apply the Generative Adversarial Networks based techniques to T2IVAE training strategy, aiming to reduce the overfitting. We evaluate our model performance on three datasets, and compare it with other several popular models using F1 score. T2IVAE achieves 0.639 on Numenta Anomaly Benchmark, 0.651 on public dataset from NASA, and 0.504 on our dataset collected from real-world scenario, outperforms other comparison models.

cs.LG↗