Searcharxiv⌕ Search

arXiv subjects

Liang Xu

Publications and source records attributed to Liang Xu.

At least 109 records · Page 6Linked to original sources
arXiv2022

Hamiltonian Deep Neural Networks Guaranteeing Non-vanishing Gradients by Design

Deep Neural Networks (DNNs) training can be difficult due to vanishing and exploding gradients during weight optimization through backpropagation. To address this problem, we propose a general class of Hamiltonian DNNs (H-DNNs) that stem from the discretization of continuous-time Hamiltonian systems and include several existing DNN architectures based on ordinary differential equations. Our main result is that a broad set of H-DNNs ensures non-vanishing gradients by design for an arbitrary network depth. This is obtained by proving that, using a semi-implicit Euler discretization scheme, the backward sensitivity matrices involved in gradient computations are symplectic. We also provide an upper-bound to the magnitude of sensitivity matrices and show that exploding gradients can be controlled through regularization. Finally, we enable distributed implementations of backward and forward propagation algorithms in H-DNNs by characterizing appropriate sparsity constraints on the weight matrices. The good performance of H-DNNs is demonstrated on benchmark classification problems, including image classification with the MNIST dataset.

cs.LG↗
arXiv2022

ActFormer: A GAN-based Transformer towards General Action-Conditioned 3D Human Motion Generation

We present a GAN-based Transformer for general action-conditioned 3D human motion generation, including not only single-person actions but also multi-person interactive actions. Our approach consists of a powerful Action-conditioned motion TransFormer (ActFormer) under a GAN training scheme, equipped with a Gaussian Process latent prior. Such a design combines the strong spatio-temporal representation capacity of Transformer, superiority in generative modeling of GAN, and inherent temporal correlations from the latent prior. Furthermore, ActFormer can be naturally extended to multi-person motions by alternately modeling temporal correlations and human interactions with Transformer encoders. To further facilitate research on multi-person motion generation, we introduce a new synthetic dataset of complex multi-person combat behaviors. Extensive experiments on NTU-13, NTU RGB+D 120, BABEL and the proposed combat dataset show that our method can adapt to various human motion representations and achieve superior performance over the state-of-the-art methods on both single-person and multi-person motion generation tasks, demonstrating a promising step towards a general human motion generator.

cs.CV↗
arXiv2022

Discovering A Variety of Objects in Spatio-Temporal Human-Object Interactions

Spatio-temporal Human-Object Interaction (ST-HOI) detection aims at detecting HOIs from videos, which is crucial for activity understanding. In daily HOIs, humans often interact with a variety of objects, e.g., holding and touching dozens of household items in cleaning. However, existing whole body-object interaction video benchmarks usually provide limited object classes. Here, we introduce a new benchmark based on AVA: Discovering Interacted Objects (DIO) including 51 interactions and 1,000+ objects. Accordingly, an ST-HOI learning task is proposed expecting vision systems to track human actors, detect interactions and simultaneously discover interacted objects. Even though today's detectors/trackers excel in object detection/tracking tasks, they perform unsatisfied to localize diverse/unseen objects in DIO. This profoundly reveals the limitation of current vision systems and poses a great challenge. Thus, how to leverage spatio-temporal cues to address object discovery is explored, and a Hierarchical Probe Network (HPN) is devised to discover interacted objects utilizing hierarchical spatio-temporal human/context cues. In extensive experiments, HPN demonstrates impressive performance. Data and code are available at https://github.com/DirtyHarryLYL/HAKE-AVA.

cs.CV↗
arXiv2022

Finding Quasars behind the Galactic Plane. II. Spectroscopic Identifications of 204 Quasars at $|b|< 20°$

Quasars behind the Galactic plane (GPQs) are important astrometric references and valuable probes of Galactic gas, yet the search for GPQs is difficult due to severe extinction and source crowding in the Galactic plane. In this paper, we present a sample of 204 spectroscopically confirmed GPQs at |b|<20°, 191 of which are new discoveries. This GPQ sample covers a wide redshift range from 0.069 to 4.487. For the subset of 230 observed GPQ candidates, the lower limit of the purity of quasars is 85.2%, and the lower limit of the fraction of stellar contaminants is 6.1%. Using a multicomponent spectral fitting, we measure the emission line and continuum flux of the GPQs, and estimate their single-epoch virial black hole masses. Due to selection effects raised from Galactic extinction and target magnitude, these GPQs have higher black hole masses and continuum luminosities in comparison to the SDSS DR7 quasar sample. The spectral-fitting results and black hole mass estimates are compiled into a main spectral catalog, and an extended spectral catalog of GPQs. The successful identifications prove the reliability of both our GPQ selection methods and the GPQ candidate catalog, shedding light on the astrometric and astrophysical programs that make use of a large sample of GPQs in the future.

astro-ph.GA↗
arXiv2022

Robust Classification using Contractive Hamiltonian Neural ODEs

Deep neural networks can be fragile and sensitive to small input perturbations that might cause a significant change in the output. In this paper, we employ contraction theory to improve the robustness of neural ODEs (NODEs). A dynamical system is contractive if all solutions with different initial conditions converge to each other exponentially fast. As a consequence, perturbations in initial conditions become less and less relevant over time. Since in NODEs the input data corresponds to the initial condition of dynamical systems, we show contractivity can mitigate the effect of input perturbations. More precisely, inspired by NODEs with Hamiltonian dynamics, we propose a class of contractive Hamiltonian NODEs (CH-NODEs). By properly tuning a scalar parameter, CH-NODEs ensure contractivity by design and can be trained using standard backpropagation. Moreover, CH-NODEs enjoy built-in guarantees of non-exploding gradients, which ensure a well-posed training process. Finally, we demonstrate the robustness of CH-NODEs on the MNIST image classification problem with noisy test data.

cs.LG↗
arXiv2022

Evaluating Modules in Graph Contrastive Learning

The recent emergence of contrastive learning approaches facilitates the application on graph representation learning (GRL), introducing graph contrastive learning (GCL) into the literature. These methods contrast semantically similar and dissimilar sample pairs to encode the semantics into node or graph embeddings. However, most existing works only performed \textbf{model-level} evaluation, and did not explore the combination space of modules for more comprehensive and systematic studies. For effective \textbf{module-level} evaluation, we propose a framework that decomposes GCL models into four modules: (1) a \textbf{sampler} to generate anchor, positive and negative data samples (nodes or graphs); (2) an \textbf{encoder} and a \textbf{readout} function to get sample embeddings; (3) a \textbf{discriminator} to score each sample pair (anchor-positive and anchor-negative); and (4) an \textbf{estimator} to define the loss function. Based on this framework, we conduct controlled experiments over a wide range of architectural designs and hyperparameter settings on node and graph classification tasks. Specifically, we manage to quantify the impact of a single module, investigate the interaction between modules, and compare the overall performance with current model architectures. Our key findings include a set of module-level guidelines for GCL, e.g., simple samplers from LINE and DeepWalk are strong and robust; an MLP encoder associated with Sum readout could achieve competitive performance on graph classification. Finally, we release our implementations and results as OpenGCL, a modularized toolkit that allows convenient reproduction, standard model and module evaluation, and easy extension. OpenGCL is available at \url{https://github.com/thunlp/OpenGCL}.

cs.LG↗
arXiv2022

Skeleton-Based Mutually Assisted Interacted Object Localization and Human Action Recognition

Skeleton data carries valuable motion information and is widely explored in human action recognition. However, not only the motion information but also the interaction with the environment provides discriminative cues to recognize the action of persons. In this paper, we propose a joint learning framework for mutually assisted "interacted object localization" and "human action recognition" based on skeleton data. The two tasks are serialized together and collaborate to promote each other, where preliminary action type derived from skeleton alone helps improve interacted object localization, which in turn provides valuable cues for the final human action recognition. Besides, we explore the temporal consistency of interacted object as constraint to better localize the interacted object with the absence of ground-truth labels. Extensive experiments on the datasets of SYSU-3D, NTU60 RGB+D, Northwestern-UCLA and UAV-Human show that our method achieves the best or competitive performance with the state-of-the-art methods for human action recognition. Visualization results show that our method can also provide reasonable interacted object localization results.

cs.CV↗
arXiv2022

Fractal superconducting nanowires detect infrared single photons with 84% system detection efficiency, 1.02 polarization sensitivity, and 20.8 ps timing resolution

The near-unity system detection efficiency (SDE) and excellent timing resolution of superconducting nanowire single-photon detectors (SNSPDs), combined with their other merits, have enabled many classical and quantum photonic applications. However, the prevalent design based on meandering nanowires makes SDE dependent on the polarization states of the incident photons; for unpolarized light, the major merit of high SDE would get compromised, which could be detrimental for photon-starved applications. Here, we create SNSPDs with an arced fractal geometry that almost completely eliminates this polarization dependence of the SDE, and we experimentally demonstrate 84$\pm$3$\%$ SDE, 1.02$^{+0.06}_{-0.02}$ polarization sensitivity at the wavelength of 1575 nm, and 20.8 ps timing jitter in a 0.1-W closed-cycle Gifford-McMahon cryocooler, at the base temperature of 2.0 K. This demonstration provides a novel, practical device structure of SNSPDs, allowing for operation in the visible, near-, and mid-infrared spectral ranges, and paves the way for polarization-insensitive single-photon detection with high SDE and high timing resolution

quant-ph↗
arXiv2022

Neural Energy Casimir Control for Port-Hamiltonian Systems

The energy Casimir method is an effective controller design approach to stabilize port-Hamiltonian systems at a desired equilibrium. However, its application relies on the availability of suitable Casimir and Lyapunov functions, whose computation are generally intractable. In this paper, we propose a neural network-based framework to learn these functions. We show how to achieve equilibrium assignment by adding suitable regularization terms in the training cost. We also propose a parameterization of Casimir functions for reducing the training complexity. Moreover, the distance between the equilibrium of the learned Lyapunov function and the desired equilibrium is analyzed, which indicates that for small suboptimality gaps, the distance decreases linearly with respect to the training loss. Our methods are backed up by simulations on a pendulum system.

eess.SY↗
arXiv2022

Finite-sample-based Spectral Radius Estimation and Stabilizability Test for Networked Control Systems

In the analysis and control of discrete-time linear time-invariant systems, the spectral radius of the system state matrix plays an essential role. Usually, it is assumed that system matrices are known, from which the spectral radius can be directly computed. Instead, we consider the setting where the system is affected by process noise, and one has only finitely many samples of system input and state measurements. We provide two methods for estimating the spectral radius and derive error bounds that hold with high probability. Moreover, we show how to use the derived results to test stabilizability for networked control systems (NCSs) with lossy channels when only finitely many samples of the system input, state, and packet drop sequence are available.

math.OC↗
arXiv2022

Formation mechanism of the rotating spoke in partially magnetized plasmas

Rotating spokes commonly occur in partially magnetized plasmas devices. In this paper, the driving mechanism behind the formation of an m=1 rotating spoke mode in a magnetically enhanced hollow cathode arc discharge is investigated by means of 2D radial-azimuthal particle-in-cell/Monte Carlo collision simulations with a uniform axial magnetic field. We find that the formation of the spoke potential hump region can be explained as a result of the positive anode sheath collapse due to the lower hybrid type instability evolving into the long wavelength regime. It is shown that an initial short-wavelength instability in the non-neutral anode sheath undergoes a sequence of transitions into the large scale modes. The sheath non-neutrality effect on the instability is considered and incorporated in the two-fluid linear theory of the lower hybrid instability. The unstable modes predicted by the theory in the linear phase and nonlinear evolution are in good agreement with the fluctuation modes developed in the particle simulations.

physics.plasm-ph↗
arXiv2022

The Observability in Unobservable Systems

In this paper, we introduce the concept of observability of targeted state variables for systems that may not be fully observable. For their estimation, we introduce and exemplify a deep filter, which is a neural network specifically designed for the estimation of targeted state variables without computing the trajectory of the entire system. The observability definition is quantitative rather than a yes or no answer so that one can compare the level of observability between different sensor locations.

eess.SY↗
arXiv2021

Data-Driven Computational Methods for the Domain of Attraction and Zubov's Equation

This paper deals with a special type of Lyapunov functions, namely the solution of Zubov's equation. Such a function can be used to characterize the domain of attraction for systems of ordinary differential equations. We derive and prove an integral form solution to Zubov's equation. For numerical computation, we develop two data-driven methods. One is based on the integration of an augmented system of differential equations; and the other one is based on deep learning. The former is effective for systems with a relatively low state space dimension and the latter is developed for high dimensional problems. The deep learning method is applied to a New England 10-generator power system model. We prove that a neural network approximation exists for the Lyapunov function of power systems such that the approximation error is a cubic polynomial of the number of generators. The error convergence rate as a function of n, the number of neurons, is proved.

math.DS↗
arXiv2021

Directly Characterizing the Coherence of Quantum Detectors by Sequential Measurement

The quantum properties of quantum measurements are indispensable resources in quantum information processing and have drawn extensive research interest. The conventional approach to reveal the quantum properties relies on the reconstruction of the entire measurement operators by quantum detector tomography. However, many specific properties can be determined by a part of matrix entries of the measurement operators, which provides us the possibility to simplify the process of property characterization. Here, we propose a general framework to directly obtain individual matrix entries of the measurement operators by sequentially measuring two non-compatible observables. This method allows us to circumvent the complete tomography of the quantum measurement and extract the useful information for our purpose. We experimentally implement this scheme to monitor the coherent evolution of a general quantum measurement by determining the off-diagonal matrix entries. The investigation of the measurement precision indicates the good feasibility of our protocol to the arbitrary quantum measurements. Our results pave the way for revealing the quantum properties of quantum measurements by selectively determining the matrix entries of the measurement operators.

quant-ph↗
arXiv2021

DataCLUE: A Benchmark Suite for Data-centric NLP

Data-centric AI has recently proven to be more effective and high-performance, while traditional model-centric AI delivers fewer and fewer benefits. It emphasizes improving the quality of datasets to achieve better model performance. This field has significant potential because of its great practicability and getting more and more attention. However, we have not seen significant research progress in this field, especially in NLP. We propose DataCLUE, which is the first Data-Centric benchmark applied in NLP field. We also provide three simple but effective baselines to foster research in this field (improve Macro-F1 up to 5.7% point). In addition, we conduct comprehensive experiments with human annotators and show the hardness of DataCLUE. We also try an advanced method: the forgetting informed bootstrapping label correction method. All the resources related to DataCLUE, including datasets, toolkit, leaderboard, and baselines, is available online at https://github.com/CLUEbenchmark/DataCLUE

cs.CL↗
arXiv2021

Attosecond delays of high harmonic emissions from isotopes of molecular hydrogen measured by Gouy phase XUV interferometer

High harmonic spectroscopy can access structural and dynamical information on molecular systems encoded in amplitude and phase of high harmonic generation (HHG) signals4. However, measurement of the harmonic phase is a daunting task. Here we present a precise measurement of HHG phase difference between two isotopes of molecular hydrogen using the advanced extreme-ultraviolet (XUV) Gouy phase interferometer. The measured phase difference is about 200 mrad, corresponding to 3 attoseconds (1 as = 10^-18 s) time delay which is nearly independent of harmonic order. The measurements agree very well with numerical calculations of a four-dimensional time-dependent Schroedinger equation. Numerical simulations also reveal the effects of molecular orientation and intra-molecular two-centre interference on the measured phase difference. This technique opens a new avenue for measuring the phase of harmonic emission for different atoms and molecules. Together with isomeric or isotopic comparisons it also enables the observation of subtle effects of molecular structures and nuclear motion on electron dynamics in strong laser fields.

physics.atom-ph↗
arXiv2021

Direct Characterization of Quantum Measurements using Weak Values

The time-symmetric formalism endows the weak measurement and its outcome, the weak value,many unique features. In particular, it allows a direct tomography of quantum states without resort to complicated reconstruction algorithms and provides an operational meaning to wave functions and density matrices. Here, we propose and experimentally demonstrate the direct tomography of a measurement apparatus by taking the backward direction of weak measurement formalism. Our protocol works rigorously with the arbitrary measurement strength, which offers an improved accuracy and precision. The precision can be further improved by taking into account the completeness condition of the measurement operators, which also ensures the feasibility of our protocol for the characterization of the arbitrary quantum measurement. Our work provides new insight on the symmetry between quantum states and measurements, as well as an efficient method to characterize a measurement apparatus.

quant-ph↗