arXiv subjects

Xiang Zheng

Publications and source records attributed to Xiang Zheng.

65 records · Page 4Linked to original sources

arXiv2023

A Policy Optimization Method Towards Optimal-time Stability

In current model-free reinforcement learning (RL) algorithms, stability criteria based on sampling methods are commonly utilized to guide policy optimization. However, these criteria only guarantee the infinite-time convergence of the system's state to an equilibrium point, which leads to sub-optimality of the policy. In this paper, we propose a policy optimization technique incorporating sampling-based Lyapunov stability. Our approach enables the system's state to reach an equilibrium point within an optimal time and maintain stability thereafter, referred to as "optimal-time stability". To achieve this, we integrate the optimization method into the Actor-Critic framework, resulting in the development of the Adaptive Lyapunov-based Actor-Critic (ALAC) algorithm. Through evaluations conducted on ten robotic tasks, our approach outperforms previous studies significantly, effectively guiding the system to generate stable patterns.

cs.RO↗

arXiv2023

CIM: Constrained Intrinsic Motivation for Sparse-Reward Continuous Control

Intrinsic motivation is a promising exploration technique for solving reinforcement learning tasks with sparse or absent extrinsic rewards. There exist two technical challenges in implementing intrinsic motivation: 1) how to design a proper intrinsic objective to facilitate efficient exploration; and 2) how to combine the intrinsic objective with the extrinsic objective to help find better solutions. In the current literature, the intrinsic objectives are all designed in a task-agnostic manner and combined with the extrinsic objective via simple addition (or used by itself for reward-free pre-training). In this work, we show that these designs would fail in typical sparse-reward continuous control tasks. To address the problem, we propose Constrained Intrinsic Motivation (CIM) to leverage readily attainable task priors to construct a constrained intrinsic objective, and at the same time, exploit the Lagrangian method to adaptively balance the intrinsic and extrinsic objectives via a simultaneous-maximization framework. We empirically show, on multiple sparse-reward continuous control tasks, that our CIM approach achieves greatly improved performance and sample efficiency over state-of-the-art methods. Moreover, the key techniques of our CIM can also be plugged into existing methods to boost their performances.

cs.LG↗

arXiv2022

Reinforcement Learning with Prior Policy Guidance for Motion Planning of Dual-Arm Free-Floating Space Robot

Reinforcement learning methods as a promising technique have achieved superior results in the motion planning of free-floating space robots. However, due to the increase in planning dimension and the intensification of system dynamics coupling, the motion planning of dual-arm free-floating space robots remains an open challenge. In particular, the current study cannot handle the task of capturing a non-cooperative object due to the lack of the pose constraint of the end-effectors. To address the problem, we propose a novel algorithm, EfficientLPT, to facilitate RL-based methods to improve planning accuracy efficiently. Our core contributions are constructing a mixed policy with prior knowledge guidance and introducing infinite norm to build a more reasonable reward function. Furthermore, our method successfully captures a rotating object with different spinning speeds.

cs.RO↗

arXiv2022

A Learning System for Motion Planning of Free-Float Dual-Arm Space Manipulator towards Non-Cooperative Object

Recent years have seen the emergence of non-cooperative objects in space, like failed satellites and space junk. These objects are usually operated or collected by free-float dual-arm space manipulators. Thanks to eliminating the difficulties of modeling and manual parameter-tuning, reinforcement learning (RL) methods have shown a more promising sign in the trajectory planning of space manipulators. Although previous studies demonstrate their effectiveness, they cannot be applied in tracking dynamic targets with unknown rotation (non-cooperative objects). In this paper, we proposed a learning system for motion planning of free-float dual-arm space manipulator (FFDASM) towards non-cooperative objects. Specifically, our method consists of two modules. Module I realizes the multi-target trajectory planning for two end-effectors within a large target space. Next, Module II takes as input the point clouds of the non-cooperative object to estimate the motional property, and then can predict the position of target points on an non-cooperative object. We leveraged the combination of Module I and Module II to track target points on a spinning object with unknown regularity successfully. Furthermore, the experiments also demonstrate the scalability and generalization of our learning system.

cs.RO↗

arXiv2020

Clean-Label Backdoor Attacks on Video Recognition Models

Deep neural networks (DNNs) are vulnerable to backdoor attacks which can hide backdoor triggers in DNNs by poisoning training data. A backdoored model behaves normally on clean test images, yet consistently predicts a particular target class for any test examples that contain the trigger pattern. As such, backdoor attacks are hard to detect, and have raised severe security concerns in real-world applications. Thus far, backdoor research has mostly been conducted in the image domain with image classification models. In this paper, we show that existing image backdoor attacks are far less effective on videos, and outline 4 strict conditions where existing attacks are likely to fail: 1) scenarios with more input dimensions (eg. videos), 2) scenarios with high resolution, 3) scenarios with a large number of classes and few examples per class (a "sparse dataset"), and 4) attacks with access to correct labels (eg. clean-label attacks). We propose the use of a universal adversarial trigger as the backdoor trigger to attack video recognition models, a situation where backdoor attacks are likely to be challenged by the above 4 strict conditions. We show on benchmark video datasets that our proposed backdoor attack can manipulate state-of-the-art video models with high success rates by poisoning only a small proportion of training data (without changing the labels). We also show that our proposed backdoor attack is resistant to state-of-the-art backdoor defense/detection methods, and can even be applied to improve image backdoor attacks. Our proposed video backdoor attack not only serves as a strong baseline for improving the robustness of video models, but also provides a new perspective for more understanding more powerful backdoor attacks.

cs.CV↗

arXiv2016

Measurement of the transfer function for a spoke cavity of C-ADS Injector I

The spoke cavities mounted in the China Accelerator Driven sub-critical System (C-ADS) have high quality factor(Q) and very small bandwidth, making them very sensitive to mechanical perturbations whether external or self-induced. The transfer function is used to characterize the response of the cavity eigen frequency to the perturbations. This paper describes a method to measure the transfer function of a spoke cavity. The measured Lorentz transfer function shows there are 206 Hz and 311 Hz mechanical eigenmodes excited by Lorentz force in the cavity of C-ADS, and the measured piezo fast tuner transfer function shows there are 12 mechanical eigenmodes from 0 to 500 Hz. According to these results, some effective measures have been taken to weaken the influence from helium pressure fluctuation, avoid mechanical resonances and improve the reliability of RF system.

physics.acc-ph↗

arXiv2016

Dilute magnetism and vibrational entropy in Fe2Al5

Fe2Al5 contains a Fe-Al matrix through which are threaded disordered one-dimensional chains of overlapping Al sites. We report magnetic, nuclear-magnetic-resonance (NMR), and specific-heat measurements addressing its magnetic and vibrational properties. The Curie-type susceptibility is found to be due to dilute moments, likely due to wrong-site Fe atoms. 27Al NMR shift and spin-lattice relaxation measurements confirm these to be indirectly coupled through a Ruderman-Kittel-Kasuya-Yoshida-type interaction. Specific-heat results indicate a large density of low-energy vibrational modes. These excitations generate a linear-T contribution to the specific heat, which however freezes out below about 10 K. These results are attributed to the presence of anharmonic vibrational modes associated with the disordered structural chains.

cond-mat.mtrl-sci↗

arXiv2015

NMR relaxation and rattling phonons in type-I Ba8Ga16Sn30 clathrate

Atomic motion of guest atoms inside semiconducting clathrate cages is considered as an important source for the glasslike thermal behavior.69Ga and 71Ga Nuclear Magnetic Resonance (NMR) studies on type-I Ba8Ga16Sn30 show a clear low temperature relaxation peak attributed to the influence of Ba rattling dynamics on the framework-atom resonance, with a quadrupolar relaxation mechanism as the leading contribution. The data are analyzed using a two-phonon Raman process, according to a recent theory involving localized anharmonic oscillators. Excellent agreement is obtained using this model, with the parameters corresponding to a uniform array of localized oscillators with very large anharmonicity.

cond-mat.mtrl-sci↗

arXiv2015

Transport and thermodynamic properties under anharmonic motion in type-I Ba8Ga16Sn30 clathrate

Anharmonic guest atom oscillation has direct connection to the thermal transport and thermo- electric behavior of type-I Ba8Ga16Sn30 clathrates. This behavior can be observed through several physical properties, with for example the heat capacity providing a measure of the overall excita- tion level structure. Localized anharmonic excitations also influence the low-temperature resistivity, as we show in this paper. By combining heat capacity, transport measurements and our previous NMR relaxation results, we address the distribution of local oscillators in this material, as well as the shape of the confining potential and the excitation energies for Ba(2) ions in the cages. We also compare to the soft-potential model and other models used for similar systems. The results show good agreement between the previously deduced anharmonic rattler potential and experimental data.

cond-mat.mtrl-sci↗

arXiv2014

SINAP surface preparation processing for 500MHz superconducting cavity

This paper illustrates the design, fabrication and experiment results of surface preparation system for 500MHz superconducting cavity at Shanghai Institute of Applied Physic (SINAP). The SINAP established a set of clean room, buffered chemical polishing equipment, and high pressure ultra-pure water rinsing facility. The whole surface preparation procedure has been operated successfully and verified by the successful vertical tests of 500MHz single cell superconducting cavity.

physics.acc-ph↗

arXiv2014

Studies of LL-type 500MHz 5-cell superconducting cavity at SINAP

A low loss (LL) type 500 MHz 5-cell superconducting niobium prototype cavity with large beam aperture has been developed successfully including the optimization, the deep drawing and electron beam welding, the surface treatment and the vertical testing. The performance of the fundamental mode was optimized and the higher order modes were damped by adopting an enlarged beam pipe for propagation. Surface preparation or treatment including mechanical polishing, buffered chemical polishing and high pressure rinsing with ultra-pure water and so on was carried out carefully to promise a perfect inner surface condition. The vertical testing results show that the accelerating voltage higher than 7.5 MV was obtained while the quality factor was better than 1E9 at 4.2 K. No obvious multipacting or field emission was found during the test. However, a quench happened while increasing the field a little higher than 7.5 MV that at present limited the cavity performance.

physics.acc-ph↗