arXiv subjects

Lanqing Li

Publications and source records attributed to Lanqing Li.

24 records · Page 2Linked to original sources

arXiv2022

Robust Imitation Learning from Corrupted Demonstrations

We consider offline Imitation Learning from corrupted demonstrations where a constant fraction of data can be noise or even arbitrary outliers. Classical approaches such as Behavior Cloning assumes that demonstrations are collected by an presumably optimal expert, hence may fail drastically when learning from corrupted demonstrations. We propose a novel robust algorithm by minimizing a Median-of-Means (MOM) objective which guarantees the accurate estimation of policy, even in the presence of constant fraction of outliers. Our theoretical analysis shows that our robust method in the corrupted setting enjoys nearly the same error scaling and sample complexity guarantees as the classical Behavior Cloning in the expert demonstration setting. Our experiments on continuous-control benchmarks validate that our method exhibits the predicted robustness and effectiveness, and achieves competitive results compared to existing imitation learning methods.

cs.LG↗

arXiv2022

DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations

AI-aided drug discovery (AIDD) is gaining increasing popularity due to its promise of making the search for new pharmaceuticals quicker, cheaper and more efficient. In spite of its extensive use in many fields, such as ADMET prediction, virtual screening, protein folding and generative chemistry, little has been explored in terms of the out-of-distribution (OOD) learning problem with \emph{noise}, which is inevitable in real world AIDD applications. In this work, we present DrugOOD, a systematic OOD dataset curator and benchmark for AI-aided drug discovery, which comes with an open-source Python package that fully automates the data curation and OOD benchmarking processes. We focus on one of the most crucial problems in AIDD: drug target binding affinity prediction, which involves both macromolecule (protein target) and small-molecule (drug compound). In contrast to only providing fixed datasets, DrugOOD offers automated dataset curator with user-friendly customization scripts, rich domain annotations aligned with biochemistry knowledge, realistic noise annotations and rigorous benchmarking of state-of-the-art OOD algorithms. Since the molecular data is often modeled as irregular graphs using graph neural network (GNN) backbones, DrugOOD also serves as a valuable testbed for \emph{graph OOD learning} problems. Extensive empirical studies have shown a significant performance gap between in-distribution and out-of-distribution experiments, which highlights the need to develop better schemes that can allow for OOD generalization under noise for AIDD.

cs.LG↗

arXiv2021

Provably Improved Context-Based Offline Meta-RL with Attention and Contrastive Learning

Meta-learning for offline reinforcement learning (OMRL) is an understudied problem with tremendous potential impact by enabling RL algorithms in many real-world applications. A popular solution to the problem is to infer task identity as augmented state using a context-based encoder, for which efficient learning of robust task representations remains an open challenge. In this work, we provably improve upon one of the SOTA OMRL algorithms, FOCAL, by incorporating intra-task attention mechanism and inter-task contrastive learning objectives, to robustify task representation learning against sparse reward and distribution shift. Theoretical analysis and experiments are presented to demonstrate the superior performance and robustness of our end-to-end and model-free framework compared to prior algorithms across multiple meta-RL benchmarks.

cs.LG↗

arXiv2021

FOCAL: Efficient Fully-Offline Meta-Reinforcement Learning via Distance Metric Learning and Behavior Regularization

We study the offline meta-reinforcement learning (OMRL) problem, a paradigm which enables reinforcement learning (RL) algorithms to quickly adapt to unseen tasks without any interactions with the environments, making RL truly practical in many real-world applications. This problem is still not fully understood, for which two major challenges need to be addressed. First, offline RL usually suffers from bootstrapping errors of out-of-distribution state-actions which leads to divergence of value functions. Second, meta-RL requires efficient and robust task inference learned jointly with control policy. In this work, we enforce behavior regularization on learned policy as a general approach to offline RL, combined with a deterministic context encoder for efficient task inference. We propose a novel negative-power distance metric on bounded context embedding space, whose gradients propagation is detached from the Bellman backup. We provide analysis and insight showing that some simple design choices can yield substantial improvements over recent approaches involving meta-RL and distance metric learning. To the best of our knowledge, our method is the first model-free and end-to-end OMRL algorithm, which is computationally efficient and demonstrated to outperform prior algorithms on several meta-RL benchmarks.

cs.LG↗

arXiv2017

A Theory of Self-Resonance After Inflation, Part 1: Adiabatic and Isocurvature Goldstone Modes

We develop a theory of self-resonance after inflation. We study a large class of models involving multiple scalar fields with an internal symmetry. For illustration, we often specialize to dimension 4 potentials, but we derive results for general potentials. This is the first part of a two part series of papers. Here in Part 1 we especially focus on the behavior of long wavelengths modes, which are found to govern most of the important physics. Since the inflaton background spontaneously breaks the time translation symmetry and the internal symmetry, we obtain Goldstone modes; these are the adiabatic and isocurvature modes. We find general conditions on the potential for when a large instability band exists for these modes at long wavelengths. For the adiabatic mode, this is determined by a sound speed derived from the time averaged potential. While for the isocurvature mode, this is determined by a speed derived from a time averaged auxiliary potential. Interestingly, we find that this instability band usually exists for one of these classes of modes, rather than both simultaneously. We focus on backgrounds that evolve radially in field space, as setup by inflation, and also mention circular orbits, as relevant to Q-balls. In Part 2 [1] we derive the central behavior from the underlying description of many particle quantum mechanics, and introduce a weak breaking of the symmetry to study corrections to particle-antiparticle production from preheating.

hep-th↗

arXiv2017

A Theory of Self-Resonance After Inflation, Part 2: Quantum Mechanics and Particle-Antiparticle Asymmetry

We further develop a theory of self-resonance after inflation in a large class of models involving multiple scalar fields. We concentrate on inflaton potentials that carry an internal symmetry, but also analyze weak breaking of this symmetry. This is the second part of a two part series of papers. Here in Part 2 we develop an understanding of the resonance structure from the underlying many particle quantum mechanics. We begin by a small amplitude analysis, which obtains the central resonant wave numbers, and relate it to perturbative processes. We show that the dominant resonance structure is determined by (i) the nonrelativistic scattering of many quantum particles and (ii) the application of Bose-Einstein statistics to the adiabatic and isocurvature modes, as introduced in Part 1 [1]. Other resonance structure is understood in terms of annihilations and decays. We setup Bunch-Davies vacuum initial conditions during inflation and track the evolution of modes including Hubble expansion. In the case of a complex inflaton carrying an internal U(1) symmetry, we show that when the isocurvature instability is active, the inflaton fragments into separate regions of ϕ-particles and anti-ϕ-particles. We then introduce a weak breaking of the U(1) symmetry; this can lead to baryogenesis, as shown by some of us recently [2,3]. Then using our results, we compute corrections to the particle-antiparticle asymmetry from this preheating era.

hep-th↗