arXiv subjects

Jian Tang

Publications and source records attributed to Jian Tang.

At least 271 records · Page 15Linked to original sources

arXiv2021

Learning Gradient Fields for Molecular Conformation Generation

We study a fundamental problem in computational chemistry known as molecular conformation generation, trying to predict stable 3D structures from 2D molecular graphs. Existing machine learning approaches usually first predict distances between atoms and then generate a 3D structure satisfying the distances, where noise in predicted distances may induce extra errors during 3D coordinate generation. Inspired by the traditional force field methods for molecular dynamics simulation, in this paper, we propose a novel approach called ConfGF by directly estimating the gradient fields of the log density of atomic coordinates. The estimated gradient fields allow directly generating stable conformations via Langevin dynamics. However, the problem is very challenging as the gradient fields are roto-translation equivariant. We notice that estimating the gradient fields of atomic coordinates can be translated to estimating the gradient fields of interatomic distances, and hence develop a novel algorithm based on recent score-based generative models to effectively estimate these gradients. Experimental results across multiple tasks show that ConfGF outperforms previous state-of-the-art baselines by a significant margin.

cs.LG↗

arXiv2021

Self-supervised Graph-level Representation Learning with Local and Global Structure

This paper studies unsupervised/self-supervised whole-graph representation learning, which is critical in many tasks such as molecule properties prediction in drug and material discovery. Existing methods mainly focus on preserving the local similarity structure between different graph instances but fail to discover the global semantic structure of the entire data set. In this paper, we propose a unified framework called Local-instance and Global-semantic Learning (GraphLoG) for self-supervised whole-graph representation learning. Specifically, besides preserving the local similarities, GraphLoG introduces the hierarchical prototypes to capture the global semantic clusters. An efficient online expectation-maximization (EM) algorithm is further developed for learning the model. We evaluate GraphLoG by pre-training it on massive unlabeled graphs followed by fine-tuning on downstream tasks. Extensive experiments on both chemical and biological benchmark data sets demonstrate the effectiveness of the proposed approach.

cs.LG↗

arXiv2021

Non-Autoregressive Electron Redistribution Modeling for Reaction Prediction

Reliably predicting the products of chemical reactions presents a fundamental challenge in synthetic chemistry. Existing machine learning approaches typically produce a reaction product by sequentially forming its subparts or intermediate molecules. Such autoregressive methods, however, not only require a pre-defined order for the incremental construction but preclude the use of parallel decoding for efficient computation. To address these issues, we devise a non-autoregressive learning paradigm that predicts reaction in one shot. Leveraging the fact that chemical reactions can be described as a redistribution of electrons in molecules, we formulate a reaction as an arbitrary electron flow and predict it with a novel multi-pointer decoding network. Experiments on the USPTO-MIT dataset show that our approach has established a new state-of-the-art top-1 accuracy and achieves at least 27 times inference speedup over the state-of-the-art methods. Also, our predictions are easier for chemists to interpret owing to predicting the electron flows.

physics.chem-ph↗

arXiv2021

An End-to-End Framework for Molecular Conformation Generation via Bilevel Programming

Predicting molecular conformations (or 3D structures) from molecular graphs is a fundamental problem in many applications. Most existing approaches are usually divided into two steps by first predicting the distances between atoms and then generating a 3D structure through optimizing a distance geometry problem. However, the distances predicted with such two-stage approaches may not be able to consistently preserve the geometry of local atomic neighborhoods, making the generated structures unsatisfying. In this paper, we propose an end-to-end solution for molecular conformation prediction called ConfVAE based on the conditional variational autoencoder framework. Specifically, the molecular graph is first encoded in a latent space, and then the 3D structures are generated by solving a principled bilevel optimization program. Extensive experiments on several benchmark data sets prove the effectiveness of our proposed approach over existing state-of-the-art approaches. Code is available at https://github.com/MinkaiXu/ConfVAE-ICML21

cs.LG↗

arXiv2021

Optically and magnetically addressable valley pseudospin of interlayer excitons in bilayer MoS2

Interlayer valley excitons in bilayer MoS2 feature concurrently large oscillator strength and long lifetime, and hence represent an advantageous scenario for valleytronic applications. However, control of valley pseudospin of interlayer excitons in pristine bilayer MoS2, which lies at the heart of valleytronics, has remained elusive. Here we report the observation of highly circularly polarized photoluminescence from interlayer excitons of bilayer MoS2 with both optical and magnetic addressability. Under excitation of circularly polarized light near exciton resonance, interlayer excitons of bilayer MoS2 show a near-unity, but negative circular polarization. Significantly, by breaking time-reversal symmetry with an out-of-plane magnetic field, a record level of spontaneous valley polarization (7.7%/Tesla) is identified for interlayer excitons in bilayer MoS2. The giant valley polarization of the interlayer excitons in bilayer MoS2, together with the feasibility of electrical/optical/magnetic control and strong oscillator strength, provides a firm basis for the development of next-generation electronic and optoelectronic applications.

cond-mat.mes-hall↗

arXiv2021

Robust Unsupervised Video Anomaly Detection by Multi-Path Frame Prediction

Video anomaly detection is commonly used in many applications such as security surveillance and is very challenging.A majority of recent video anomaly detection approaches utilize deep reconstruction models, but their performance is often suboptimal because of insufficient reconstruction error differences between normal and abnormal video frames in practice. Meanwhile, frame prediction-based anomaly detection methods have shown promising performance. In this paper, we propose a novel and robust unsupervised video anomaly detection method by frame prediction with proper design which is more in line with the characteristics of surveillance videos. The proposed method is equipped with a multi-path ConvGRU-based frame prediction network that can better handle semantically informative objects and areas of different scales and capture spatial-temporal dependencies in normal videos. A noise tolerance loss is introduced during training to mitigate the interference caused by background noise. Extensive experiments have been conducted on the CUHK Avenue, ShanghaiTech Campus, and UCSD Pedestrian datasets, and the results show that our proposed method outperforms existing state-of-the-art approaches. Remarkably, our proposed method obtains the frame-level AUROC score of 88.3% on the CUHK Avenue dataset.

cs.CV↗

arXiv2021

Learning Dynamic Belief Graphs to Generalize on Text-Based Games

Playing text-based games requires skills in processing natural language and sequential decision making. Achieving human-level performance on text-based games remains an open challenge, and prior research has largely relied on hand-crafted structured representations and heuristics. In this work, we investigate how an agent can plan and generalize in text-based games using graph-structured representations learned end-to-end from raw text. We propose a novel graph-aided transformer agent (GATA) that infers and updates latent belief graphs during planning to enable effective action selection by capturing the underlying game dynamics. GATA is trained using a combination of reinforcement and self-supervised learning. Our work demonstrates that the learned graph-based representations help agents converge to better policies than their text-only counterparts and facilitate effective generalization across game configurations. Experiments on 500+ unique games from the TextWorld suite show that our best agent outperforms text-based baselines by an average of 24.2%.

cs.CL↗

arXiv2021

GRADE: Graph Dynamic Embedding

Representation learning of static and more recently dynamically evolving graphs has gained noticeable attention. Existing approaches for modelling graph dynamics focus extensively on the evolution of individual nodes independently of the evolution of mesoscale community structures. As a result, current methods do not provide useful tools to study and cannot explicitly capture temporal community dynamics. To address this challenge, we propose GRADE - a probabilistic model that learns to generate evolving node and community representations by imposing a random walk prior over their trajectories. Our model also learns node community membership which is updated between time steps via a transition matrix. At each time step link generation is performed by first assigning node membership from a distribution over the communities, and then sampling a neighbor from a distribution over the nodes for the assigned community. We parametrize the node and community distributions with neural networks and learn their parameters via variational inference. Experiments demonstrate GRADE outperforms baselines in dynamic link prediction, shows favourable performance on dynamic community detection, and identifies coherent and interpretable evolving communities.

cs.LG↗

arXiv2021

Learning Neural Generative Dynamics for Molecular Conformation Generation

We study how to generate molecule conformations (i.e., 3D structures) from a molecular graph. Traditional methods, such as molecular dynamics, sample conformations via computationally expensive simulations. Recently, machine learning methods have shown great potential by training on a large collection of conformation data. Challenges arise from the limited model capacity for capturing complex distributions of conformations and the difficulty in modeling long-range dependencies between atoms. Inspired by the recent progress in deep generative models, in this paper, we propose a novel probabilistic framework to generate valid and diverse conformations given a molecular graph. We propose a method combining the advantages of both flow-based and energy-based models, enjoying: (1) a high model capacity to estimate the multimodal conformation distribution; (2) explicitly capturing the complex long-range dependencies between atoms in the observation space. Extensive experiments demonstrate the superior performance of the proposed method on several benchmarks, including conformation generation and distance modeling tasks, with a significant improvement over existing generative models for molecular conformation sampling.

cs.LG↗

arXiv2021

Constraining sterile neutrinos by core-collapse supernovae with multiple detectors

The eV-scale sterile neutrino has been proposed to explain some anomalous results in experiments, \textit{such as} the deficit of reactor neutrino fluxes and the excess of $\barν_μ\to\barν_e$ in LSND. This hypothesis can be tested by future core-collapse supernova neutrino detection independently since the active-sterile mixing scheme affects the flavor conversion of neutrinos inside the supernova. In this work, we compute the predicted supernova neutrino events in future detectors -- DUNE, Hyper-K, and JUNO -- for neutrinos emitted during the neutronization burst phase when the luminosity of $ν_e$ dominates the other flavors. We find that for a supernova occurring within 10 kpc, the difference in the event numbers with and without sterile neutrinos allows to exclude the sterile neutrino hypothesis at more than $99\%$ confidence level robustly. The derived constraints on sterile neutrinos mixing parameters are comparably better than the results from cosmology and on-going or proposed reactor experiments by more than two orders of magnitude in the $\sin^22θ_{14}$-$Δm_{41}^2$ plane.

hep-ph↗

arXiv2021

Emergence of Chern insulating states in non-Magic angle twisted bilayer graphene

Twisting two layers into a magic angle (MA) of ~1.1° is found essential to create low energy flat bands and the resulting correlated insulating, superconducting, and magnetic phases in twisted bilayer graphene (TBG). While most of previous works focus on revealing these emergent states in MA-TBG, a study of the twist angle dependence, which helps to map an evolution of these phases, is yet less explored. Here, we report a magneto-transport study on one non-magic angle TBG device, whose twist angle θ changes from 1.25° at one end to 1.43° at the other. For θ=1.25°, we observe an emergence of topological insulating states at hole side with a sequence of Chern number |C|=4-|v|, where v is the number of electrons (holes) in moiré unite cell. When θ>1.25°, the Chern insulator from flat band disappears and evolves into fractal Hofstadter butterfly quantum Hall insulator where magnetic flux in one moiré unite cell matters. Our observations will stimulate further theoretical and experimental investigations on the relationship between electron interactions and non-trivial band topology.

cond-mat.mes-hall↗

arXiv2021

Utilising Graph Machine Learning within Drug Discovery and Development

Graph Machine Learning (GML) is receiving growing interest within the pharmaceutical and biotechnology industries for its ability to model biomolecular structures, the functional relationships between them, and integrate multi-omic datasets - amongst other data types. Herein, we present a multidisciplinary academic-industrial review of the topic within the context of drug discovery and development. After introducing key terms and modelling approaches, we move chronologically through the drug development pipeline to identify and summarise work incorporating: target identification, design of small molecules and biologics, and drug repurposing. Whilst the field is still emerging, key milestones including repurposed drugs entering in vivo studies, suggest graph machine learning will become a modelling framework of choice within biomedical machine learning.

q-bio.QM↗

arXiv2021

Non-autoregressive electron flow generation for reaction prediction

Reaction prediction is a fundamental problem in computational chemistry. Existing approaches typically generate a chemical reaction by sampling tokens or graph edits sequentially, conditioning on previously generated outputs. These autoregressive generating methods impose an arbitrary ordering of outputs and prevent parallel decoding during inference. We devise a novel decoder that avoids such sequential generating and predicts the reaction in a Non-Autoregressive manner. Inspired by physical-chemistry insights, we represent edge edits in a molecule graph as electron flows, which can then be predicted in parallel. To capture the uncertainty of reactions, we introduce latent variables to generate multi-modal outputs. Following previous works, we evaluate our model on USPTO MIT dataset. Our model achieves both an order of magnitude lower inference latency, with state-of-the-art top-1 accuracy and comparable performance on Top-K sampling.

physics.chem-ph↗

arXiv2021

Probing the doubly-charged Higgs with Muonium to Antimuonium Conversion Experiment

The spontaneous muonium-to-antimuonium conversion is one of the interesting charged lepton flavor violation processes. MACE is the next generation experiment to probe such a phenomenon. In models with a triplet Higgs to generate neutrino masses, such as Type-II seesaw and its variant, this process can be induced by the doubly-charged Higgs contained in it. In this article, we study the prospect of MACE to probe these models via the muonium-to-antimuonium transitions. After considering the limits from $μ^+ \rightarrow e^+ γ$ and $μ^+ \rightarrow e^+ e^- e^+$, we find that MACE could probe a parameter space for the doubly-charged Higgs which is beyond the reach of LHC and other flavor experiments.

hep-ph↗

arXiv2021

Non-standard interactions in SMEFT confronted with terrestrial neutrino experiments

The Standard Model Effective Field Theory (SMEFT) provides a systematic and model-independent framework to study neutrino non-standard interactions (NSIs). We study the constraining power of the on-going neutrino oscillation experiments T2K, NO$ν$A, Daya Bay, Double Chooz and RENO in the SMEFT framework. A full consideration of matching is provided between different effective field theories and the renormalization group running at different scales, filling the gap between the low-energy neutrino oscillation experiments and SMEFT at the UV scale. We first illustrate our method with a top-down approach in a simplified scalar leptoquark model, showing more stringent constraints from the neutrino oscillation experiments compared to collider studies. We then provide a bottom-up study on individual dimension-6 SMEFT operators and find NSIs in neutrino experiments already sensitive to new physics at $\sim$20 TeV when the Wilson coefficients are fixed at unity. We also investigate the correlation among multiple operators at the UV scale and find it could change the constraints on SMEFT operators by several orders of magnitude compared with when only one operator is considered. Furthermore, we find that accelerator and reactor neutrino experiments are sensitive to different SMEFT operators, which highlights the complementarity of the two experiment types.

hep-ph↗

arXiv2021

Towards Generalized Implementation of Wasserstein Distance in GANs

Wasserstein GANs (WGANs), built upon the Kantorovich-Rubinstein (KR) duality of Wasserstein distance, is one of the most theoretically sound GAN models. However, in practice it does not always outperform other variants of GANs. This is mostly due to the imperfect implementation of the Lipschitz condition required by the KR duality. Extensive work has been done in the community with different implementations of the Lipschitz constraint, which, however, is still hard to satisfy the restriction perfectly in practice. In this paper, we argue that the strong Lipschitz constraint might be unnecessary for optimization. Instead, we take a step back and try to relax the Lipschitz constraint. Theoretically, we first demonstrate a more general dual form of the Wasserstein distance called the Sobolev duality, which relaxes the Lipschitz constraint but still maintains the favorable gradient property of the Wasserstein distance. Moreover, we show that the KR duality is actually a special case of the Sobolev duality. Based on the relaxed duality, we further propose a generalized WGAN training scheme named Sobolev Wasserstein GAN (SWGAN), and empirically demonstrate the improvement of SWGAN over existing methods with extensive experiments.

cs.LG↗

arXiv2020

XLVIN: eXecuted Latent Value Iteration Nets

Value Iteration Networks (VINs) have emerged as a popular method to incorporate planning algorithms within deep reinforcement learning, enabling performance improvements on tasks requiring long-range reasoning and understanding of environment dynamics. This came with several limitations, however: the model is not incentivised in any way to perform meaningful planning computations, the underlying state space is assumed to be discrete, and the Markov decision process (MDP) is assumed fixed and known. We propose eXecuted Latent Value Iteration Networks (XLVINs), which combine recent developments across contrastive self-supervised learning, graph representation learning and neural algorithmic reasoning to alleviate all of the above limitations, successfully deploying VIN-style models on generic environments. XLVINs match the performance of VIN-like models when the underlying MDP is discrete, fixed and known, and provides significant improvements to model-free baselines across three general MDP setups.

cs.LG↗

arXiv2020

KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation

Pre-trained language representation models (PLMs) cannot well capture factual knowledge from text. In contrast, knowledge embedding (KE) methods can effectively represent the relational facts in knowledge graphs (KGs) with informative entity embeddings, but conventional KE models cannot take full advantage of the abundant textual information. In this paper, we propose a unified model for Knowledge Embedding and Pre-trained LanguagE Representation (KEPLER), which can not only better integrate factual knowledge into PLMs but also produce effective text-enhanced KE with the strong PLMs. In KEPLER, we encode textual entity descriptions with a PLM as their embeddings, and then jointly optimize the KE and language modeling objectives. Experimental results show that KEPLER achieves state-of-the-art performances on various NLP tasks, and also works remarkably well as an inductive KE model on KG link prediction. Furthermore, for pre-training and evaluating KEPLER, we construct Wikidata5M, a large-scale KG dataset with aligned entity descriptions, and benchmark state-of-the-art KE methods on it. It shall serve as a new KE benchmark and facilitate the research on large KG, inductive KE, and KG with text. The source code can be obtained from https://github.com/THU-KEG/KEPLER.

cs.CL↗