arXiv subjects

Jian Tang

Publications and source records attributed to Jian Tang.

At least 235 records · Page 13Linked to original sources

arXiv2022

Temperature-linear Resistivity in Twisted Double Bilayer Graphene

We report an experimental study of carrier density (n), displacement field (D) and twist angle (θ) dependence of temperature (T)-linear resistivity in twisted double bilayer graphene (TDBG). For a large twist angle (θ>1.5°) where correlated insulating states are absent, we observe a T-linear resistivity (with the slope of the order ~10Ω/K) over a wide range of carrier density and its slope decreases with increasing of n, in agreement with acoustic phonon scattering model semi-quantitatively. The slope of T-linear resistivity is non-monotonically dependent on the displacement field with a single peak structure. For device with θ~1.23° at which correlated states emerge, the slope of T-linear resistivity is found maximum (~100Ω/K) at the boundary of the halo structure where phase transition occurs, with signatures of continuous phase transition, Planckian dissipation, and the diverging effective mass; these observations are in line with quantum critical behaviors, which might be due to the symmetry-breaking instability at the critical points. Our results shed new light on correlated physics in TDBG and other twisted moiré systems.

cond-mat.mes-hall↗

arXiv2022

Measurement of Muon-induced Neutron Production at the China Jinping Underground Laboratory

Solar, terrestrial, and supernova neutrino experiments are subject to muon-induced radioactive backgrounds. The China Jinping Underground Laboratory (CJPL), with its unique advantage of a 2400 m rock coverage and long distance from nuclear power plants, is ideal for MeV-scale neutrino experiments. Using a 1-ton prototype detector of the Jinping Neutrino Experiment (JNE), we detected 343 high-energy cosmic-ray muons and (7.86$ \pm $3.97) muon-induced neutrons from an 820.28-day dataset at the first phase of CJPL (CJPL-I). Based on the muon-induced neutrons, we measured the corresponding muon-induced neutron yield in a liquid scintillator to be $ (3.44 \pm 1.86_{\rm stat.}\pm 0.76_{\rm syst.})\times 10^{-4}μ^{-1}\rm g^{-1}cm^{2} $ at an average muon energy of \SI{340}{GeV}. We provided the first study for such neutron background at CJPL. A global fit including this measurement shows a power-law coefficient of (0.75$ \pm $0.02) for the dependence of the neutron yield at the liquid scintillator on muon energy.

hep-ex↗

arXiv2022

Generative Coarse-Graining of Molecular Conformations

Coarse-graining (CG) of molecular simulations simplifies the particle representation by grouping selected atoms into pseudo-beads and drastically accelerates simulation. However, such CG procedure induces information losses, which makes accurate backmapping, i.e., restoring fine-grained (FG) coordinates from CG coordinates, a long-standing challenge. Inspired by the recent progress in generative models and equivariant networks, we propose a novel model that rigorously embeds the vital probabilistic nature and geometric consistency requirements of the backmapping transformation. Our model encodes the FG uncertainties into an invariant latent space and decodes them back to FG geometries via equivariant convolutions. To standardize the evaluation of this domain, we provide three comprehensive benchmarks based on molecular dynamics trajectories. Experiments show that our approach always recovers more realistic structures and outperforms existing data-driven methods with a significant margin.

cs.LG↗

arXiv2022

Pre-training Molecular Graph Representation with 3D Geometry

Molecular graph representation learning is a fundamental problem in modern drug and material discovery. Molecular graphs are typically modeled by their 2D topological structures, but it has been recently discovered that 3D geometric information plays a more vital role in predicting molecular functionalities. However, the lack of 3D information in real-world scenarios has significantly impeded the learning of geometric graph representation. To cope with this challenge, we propose the Graph Multi-View Pre-training (GraphMVP) framework where self-supervised learning (SSL) is performed by leveraging the correspondence and consistency between 2D topological structures and 3D geometric views. GraphMVP effectively learns a 2D molecular graph encoder that is enhanced by richer and more discriminative 3D geometry. We further provide theoretical insights to justify the effectiveness of GraphMVP. Finally, comprehensive experiments show that GraphMVP can consistently outperform existing graph SSL methods.

cs.LG↗

arXiv2022

Towards Interpretable Natural Language Understanding with Explanations as Latent Variables

Recently generating natural language explanations has shown very promising results in not only offering interpretable explanations but also providing additional information and supervision for prediction. However, existing approaches usually require a large set of human annotated explanations for training while collecting a large set of explanations is not only time consuming but also expensive. In this paper, we develop a general framework for interpretable natural language understanding that requires only a small set of human annotated explanations for training. Our framework treats natural language explanations as latent variables that model the underlying reasoning process of a neural model. We develop a variational EM framework for optimization where an explanation generation module and an explanation-augmented prediction module are alternatively optimized and mutually enhance each other. Moreover, we further propose an explanation-based self-training method under this framework for semi-supervised learning. It alternates between assigning pseudo-labels to unlabeled data and generating new explanations to iteratively improve each other. Experiments on two natural language understanding tasks demonstrate that our framework can not only make effective predictions in both supervised and semi-supervised settings, but also generate good natural language explanation.

cs.CL↗

arXiv2022

HIRL: A General Framework for Hierarchical Image Representation Learning

Learning self-supervised image representations has been broadly studied to boost various visual understanding tasks. Existing methods typically learn a single level of image semantics like pairwise semantic similarity or image clustering patterns. However, these methods can hardly capture multiple levels of semantic information that naturally exists in an image dataset, e.g., the semantic hierarchy of "Persian cat to cat to mammal" encoded in an image database for species. It is thus unknown whether an arbitrary image self-supervised learning (SSL) approach can benefit from learning such hierarchical semantics. To answer this question, we propose a general framework for Hierarchical Image Representation Learning (HIRL). This framework aims to learn multiple semantic representations for each image, and these representations are structured to encode image semantics from fine-grained to coarse-grained. Based on a probabilistic factorization, HIRL learns the most fine-grained semantics by an off-the-shelf image SSL approach and learns multiple coarse-grained semantics by a novel semantic path discrimination scheme. We adopt six representative image SSL methods as baselines and study how they perform under HIRL. By rigorous fair comparison, performance gain is observed on all the six methods for diverse downstream tasks, which, for the first time, verifies the general effectiveness of learning hierarchical image semantics. All source code and model weights are available at https://github.com/hirl-team/HIRL

cs.CV↗

arXiv2022

Tyger: Task-Type-Generic Active Learning for Molecular Property Prediction

How to accurately predict the properties of molecules is an essential problem in AI-driven drug discovery, which generally requires a large amount of annotation for training deep learning models. Annotating molecules, however, is quite costly because it requires lab experiments conducted by experts. To reduce annotation cost, deep Active Learning (AL) methods are developed to select only the most representative and informative data for annotating. However, existing best deep AL methods are mostly developed for a single type of learning task (e.g., single-label classification), and hence may not perform well in molecular property prediction that involves various task types. In this paper, we propose a Task-type-generic active learning framework (termed Tyger) that is able to handle different types of learning tasks in a unified manner. The key is to learn a chemically-meaningful embedding space and perform active selection fully based on the embeddings, instead of relying on task-type-specific heuristics (e.g., class-wise prediction probability) as done in existing works. Specifically, for learning the embedding space, we instantiate a querying module that learns to translate molecule graphs into corresponding SMILES strings. Furthermore, to ensure that samples selected from the space are both representative and informative, we propose to shape the embedding space by two learning objectives, one based on domain knowledge and the other leveraging feedback from the task learner (i.e., model that performs the learning task at hand). We conduct extensive experiments on benchmark datasets of different task types. Experimental results show that Tyger consistently achieves high AL performance on molecular property prediction, outperforming baselines by a large margin. We also perform ablative experiments to verify the effectiveness of each component in Tyger.

cs.LG↗

arXiv2022

Non-minimal Lorentz invariance violation in light of muon anomalous magnetic moment and long-baseline neutrino oscillation data

In light of the increasing hints of new physics at the muon $g-2$ and neutrino oscillation experiments, we consider the recently observed tension in the long-baseline neutrino oscillation experiments as a potential indication of Lorentz invariance violation. For this purpose, the latest data from T2K and NO$ν$A is analysed in presence of non-minimal Lorentz invariance violation. Indeed, we find that isotropic violation in dimensions $D =$ 4, 5 and 6 can alleviate the tension in neutrino oscillation data by 0.4$-$2.4$σ$ CL significance, with the isotropic coefficient $γ^{(5)}_{ττ} =$ 3.58$\times$10$^{-32}$GeV$^{-1}$ yielding the best fit. At the same time, the anomalous muon $g-2$ result can be reproduced with an additional non-isotropic violation of $d^{zt} =$ -1.7$\times$10$^{-25}$. The analysis highlights the possibility of simultaneous relaxation of experimental tensions with Lorentz invariance violation of mixed nature.

hep-ph↗

arXiv2022

Constraints on cosmic-ray boosted DM in CDEX-10

Dark matter (DM) direct detection experiments have been setting strong limits on the DM-nucleon scattering cross section at the DM mass above a few GeV, but leave large parameter space unexplored in the low mass region. DM is likely to be scattered and boosted by relativistic cosmic rays in the expanding universe if it can generate nuclear recoils in direct detection experiments to offer observable signals. Since low energy threshold detectors using Germanium have provided good constraints on ordinary halo GeV-scale DM, it is necessary to re-analyze 102.8 kg$\times$day data in the CDEX-10 experiment assuming that DM is boosted by cosmic rays. For the DM mass range 1 keV $<m_χ<$ 1 MeV and the effective distance within 1 kpc, we reach an almost flat floor limit at $8.32\times10^{-30}$ cm$^2$ on spin-independent DM-nucleon scattering cross section at a 90\% confidence level. The CDEX-10 result is able to close the gap unambiguously in the parameter space between MiniBooNE and XENON1T constraints which was partially hindered by the Earth attenuation effect. We also quantitatively calculate expected neutrino floor on searching for CRBDM in future direct detection experiments using Germanium.

hep-ph↗

arXiv2022

A Roadmap for Big Model

With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm. Researchers have achieved various outcomes in the construction of BMs and the BM application in many fields. At present, there is a lack of research work that sorts out the overall progress of BMs and guides the follow-up research. In this paper, we cover not only the BM technologies themselves but also the prerequisites for BM training and applications with BMs, dividing the BM review into four parts: Resource, Models, Key Technologies and Application. We introduce 16 specific BM-related topics in those four parts, they are Data, Knowledge, Computing System, Parallel Training System, Language Model, Vision Model, Multi-modal Model, Theory&Interpretability, Commonsense Reasoning, Reliability&Security, Governance, Evaluation, Machine Translation, Text Generation, Dialogue and Protein Research. In each topic, we summarize clearly the current studies and propose some future research directions. At the end of this paper, we conclude the further development of BMs in a more general view.

cs.LG↗

arXiv2022

Weakly-supervised Temporal Path Representation Learning with Contrastive Curriculum Learning -- Extended Version

In step with the digitalization of transportation, we are witnessing a growing range of path-based smart-city applications, e.g., travel-time estimation and travel path ranking. A temporal path(TP) that includes temporal information, e.g., departure time, into the path is fundamental to enable such applications. In this setting, it is essential to learn generic temporal path representations(TPRs) that consider spatial and temporal correlations simultaneously and that can be used in different applications, i.e., downstream tasks. Existing methods fail to achieve the goal since (i) supervised methods require large amounts of task-specific labels when training and thus fail to generalize the obtained TPRs to other tasks; (ii) through unsupervised methods can learn generic representations, they disregard the temporal aspect, leading to sub-optimal results. To contend with the limitations of existing solutions, we propose a Weakly-Supervised Contrastive (WSC) learning model. We first propose a temporal path encoder that encodes both the spatial and temporal information of a temporal path into a TPR. To train the encoder, we introduce weak labels that are easy and inexpensive to obtain and are relevant to different tasks, e.g., temporal labels indicating peak vs. off-peak hours from departure times. Based on the weak labels, we construct meaningful positive and negative temporal path samples by considering both spatial and temporal information, which facilities training the encoder using contrastive learning by pulling closer to the positive samples' representations while pushing away the negative samples' representations. To better guide contrastive learning, we propose a learning strategy based on Curriculum Learning such that the learning performs from easy to hard training instances. Experiments studies verify the effectiveness of the proposed method.

cs.LG↗

arXiv2022

Neural Structured Prediction for Inductive Node Classification

This paper studies node classification in the inductive setting, i.e., aiming to learn a model on labeled training graphs and generalize it to infer node labels on unlabeled test graphs. This problem has been extensively studied with graph neural networks (GNNs) by learning effective node representations, as well as traditional structured prediction methods for modeling the structured output of node labels, e.g., conditional random fields (CRFs). In this paper, we present a new approach called the Structured Proxy Network (SPN), which combines the advantages of both worlds. SPN defines flexible potential functions of CRFs with GNNs. However, learning such a model is nontrivial as it involves optimizing a maximin game with high-cost inference. Inspired by the underlying connection between joint and marginal distributions defined by Markov networks, we propose to solve an approximate version of the optimization problem as a proxy, which yields a near-optimal solution, making learning more efficient. Extensive experiments on two settings show that our approach outperforms many competitive baselines.

cs.LG↗

arXiv2022

DAIS: Automatic Channel Pruning via Differentiable Annealing Indicator Search

The convolutional neural network has achieved great success in fulfilling computer vision tasks despite large computation overhead against efficient deployment. Structured (channel) pruning is usually applied to reduce the model redundancy while preserving the network structure, such that the pruned network can be easily deployed in practice. However, existing structured pruning methods require hand-crafted rules which may lead to tremendous pruning space. In this paper, we introduce Differentiable Annealing Indicator Search (DAIS) that leverages the strength of neural architecture search in the channel pruning and automatically searches for the effective pruned model with given constraints on computation overhead. Specifically, DAIS relaxes the binarized channel indicators to be continuous and then jointly learns both indicators and model parameters via bi-level optimization. To bridge the non-negligible discrepancy between the continuous model and the target binarized model, DAIS proposes an annealing-based procedure to steer the indicator convergence towards binarized states. Moreover, DAIS designs various regularizations based on a priori structural knowledge to control the pruning sparsity and to improve model performance. Experimental results show that DAIS outperforms state-of-the-art pruning methods on CIFAR-10, CIFAR-100, and ImageNet.

cs.CV↗

arXiv2022

Snowmass2021 Whitepaper: Muonium to antimuonium conversion

The spontaneous muonium to antimuonium conversion is one of the interesting charged lepton flavor violation processes. It serves as a clear indication of new physics and plays an important role in constraining the parameter space beyond Standard Model. MACE is a proposed experiment to probe such a phenomenon and expected to enhance the sensitivity to the conversion probability by more than two orders of magnitude from the current best upper constraint obtained by the PSI experiment two decades ago. Recent developments in the theoretical and experimental aspects to search for such a rare process are summarized.

hep-ph↗

arXiv2022

RGB-Depth Fusion GAN for Indoor Depth Completion

The raw depth image captured by the indoor depth sensor usually has an extensive range of missing depth values due to inherent limitations such as the inability to perceive transparent objects and limited distance range. The incomplete depth map burdens many downstream vision tasks, and a rising number of depth completion methods have been proposed to alleviate this issue. While most existing methods can generate accurate dense depth maps from sparse and uniformly sampled depth maps, they are not suitable for complementing the large contiguous regions of missing depth values, which is common and critical. In this paper, we design a novel two-branch end-to-end fusion network, which takes a pair of RGB and incomplete depth images as input to predict a dense and completed depth map. The first branch employs an encoder-decoder structure to regress the local dense depth values from the raw depth map, with the help of local guidance information extracted from the RGB image. In the other branch, we propose an RGB-depth fusion GAN to transfer the RGB image to the fine-grained textured depth map. We adopt adaptive fusion modules named W-AdaIN to propagate the features across the two branches, and we append a confidence fusion head to fuse the two outputs of the branches for the final depth map. Extensive experiments on NYU-Depth V2 and SUN RGB-D demonstrate that our proposed method clearly improves the depth completion performance, especially in a more realistic setting of indoor environments with the help of the pseudo depth map.

cs.CV↗

arXiv2022

Implications of Topological Imbalance for Representation Learning on Biomedical Knowledge Graphs

Adoption of recently developed methods from machine learning has given rise to creation of drug-discovery knowledge graphs (KG) that utilize the interconnected nature of the domain. Graph-based modelling of the data, combined with KG embedding (KGE) methods, are promising as they provide a more intuitive representation and are suitable for inference tasks such as predicting missing links. One common application is to produce ranked lists of genes for a given disease, where the rank is based on the perceived likelihood of association between the gene and the disease. It is thus critical that these predictions are not only pertinent but also biologically meaningful. However, KGs can be biased either directly due to the underlying data sources that are integrated or due to modeling choices in the construction of the graph, one consequence of which is that certain entities can get topologically overrepresented. We demonstrate the effect of these inherent structural imbalances, resulting in densely-connected entities being highly ranked no matter the context. We provide support for this observation across different datasets, models as well as predictive tasks. Further, we present various graph perturbation experiments which yield more support to the observation that KGE models can be more influenced by the frequency of entities rather than any biological information encoded within the relations. Our results highlight the importance of data modeling choices, and emphasizes the need for practitioners to be mindful of these issues when interpreting model outputs and during KG composition.

cs.LG↗

arXiv2022

Layer-by-Layer Epitaxy of Multilayer MoS2 Wafers

Two-dimensional (2D) semiconductor of MoS2 has great potential for advanced electronics technologies beyond silicon1-9. So far, high-quality monolayer MoS2 wafers10-12 are already available and various demonstrations from individual transistors to integrated circuits have also been shown13-15. In addition to the monolayer, multilayers have narrower band gaps but improved carrier mobilities and current capacities over the monolayer5,16-18. However, achieving high-quality multilayer MoS2 wafers remains a challenge. Here we report the growth of high quality multilayer MoS2 4-inch wafers via the layer-by-layer epitaxy process. The epitaxy leads to well-defined stacking orders between adjacent epitaxial layers and offers a delicate control of layer numbers up to 6. Systematic evaluations on the atomic structures and electronic properties were carried out for achieved wafers with different layer numbers. Significant improvements on device performances were found in thicker-layer field effect transistors (FETs), as expected. For example, the average field-effect mobility (μFE) at room temperature (RT) can increase from ~80 cm2V-1s-1 for monolayer to ~110/145 cm2V-1s-1 for bilayer/trilayer devices. The highest RT μFE=234.7 cm2V-1s-1 and a record-high on-current densities of 1.704 mAμm-1 at Vds=2 V were also achieved in trilayer MoS2 FETs with a high on/off ratio exceeding 107. Our work hence moves a step closer to practical applications of 2D MoS2 in electronics.

physics.app-ph↗

arXiv2022

Promising Technologies and R&D Directions for the Future Muon Collider Detectors

Among the post-LHC generation of particle accelerators, the muon collider represents a unique machine with capability to provide very high energy leptonic collisions and to open the path to a vast and mostly unexplored physics programme. However, on the experimental side, such great physics potential is accompanied by unprecedented technological challenges, due to the fact that muons are unstable particles. Their decay products interact with the machine elements and produce an intense flux of background particles that eventually reach the detector and may degrade its performance. In this paper, we present technologies that have a potential to match the challenging specifications of a muon collider detector and outline a path forward for the future R&D efforts.

physics.ins-det↗