arXiv subjects

Zhao Li

Publications and source records attributed to Zhao Li.

At least 73 records · Page 4Linked to original sources

arXiv2023

Analytic three-loop QCD corrections to top-quark and semileptonic $b\to u$ decays

We present the first analytic results of N$^3$LO QCD corrections to the top-quark decay width. We focus on the dominant leading color contribution, which includes light-quark loops. At NNLO, this dominant contribution accounts for 95% of the total correction. By utilizing the optical theorem, the N$^3$LO corrections are related to the imaginary parts of the four-loop self-energy Feynman diagrams, which are calculated with differential equations. The results are expressed in terms of harmonic polylogarithms, enabling fast and accurate evaluation. The third-order QCD corrections decrease the LO decay width by 0.667%, and the scale uncertainty is reduced by half compared to the NNLO result. The most precise prediction for the top-quark width is now 1.321 GeV for $m_t=172.69$ GeV. Additionally, we obtain the third-order QCD corrections to the dilepton invariant mass spectrum and decay width in the semileptonic $b\to u$ transition.

hep-ph↗

arXiv2023

Pre-trained Language Models in Biomedical Domain: A Systematic Survey

Pre-trained language models (PLMs) have been the de facto paradigm for most natural language processing (NLP) tasks. This also benefits biomedical domain: researchers from informatics, medicine, and computer science (CS) communities propose various PLMs trained on biomedical datasets, e.g., biomedical text, electronic health records, protein, and DNA sequences for various biomedical tasks. However, the cross-discipline characteristics of biomedical PLMs hinder their spreading among communities; some existing works are isolated from each other without comprehensive comparison and discussions. It expects a survey that not only systematically reviews recent advances of biomedical PLMs and their applications but also standardizes terminology and benchmarks. In this paper, we summarize the recent progress of pre-trained language models in the biomedical domain and their applications in biomedical downstream tasks. Particularly, we discuss the motivations and propose a taxonomy of existing biomedical PLMs. Their applications in biomedical downstream tasks are exhaustively discussed. At last, we illustrate various limitations and future trends, which we hope can provide inspiration for the future research of the research community.

cs.CL↗

arXiv2023

A Generative Hypergraph Model for Double Heterogeneity

While network science has become an indispensable tool for studying complex systems, the conventional use of pairwise links often shows limitations in describing high-order interactions properly. Hypergraphs, where each edge can connect more than two nodes, have thus become a new paradigm in network science. Yet, we are still in lack of models linking network growth and hyperedge expansion, both of which are commonly observable in the real world. Here, we propose a generative hypergraph model by employing the preferential attachment mechanism in both nodes and hyperedge formation. The model can produce bi-heterogeneity, exhibiting scale-free distributions in both hyperdegree and hyperedge size. We provide a mean-field treatment that gives the expression of the two scaling exponents, which agree with the numerical simulations. Our model may help to understand the networked systems showing both types of heterogeneity and facilitate the study of complex dynamics thereon.

physics.soc-ph↗

arXiv2023

Cold-Start based Multi-Scenario Ranking Model for Click-Through Rate Prediction

Online travel platforms (OTPs), e.g., Ctrip.com or Fliggy.com, can effectively provide travel-related products or services to users. In this paper, we focus on the multi-scenario click-through rate (CTR) prediction, i.e., training a unified model to serve all scenarios. Existing multi-scenario based CTR methods struggle in the context of OTP setting due to the ignorance of the cold-start users who have very limited data. To fill this gap, we propose a novel method named Cold-Start based Multi-scenario Network (CSMN). Specifically, it consists of two basic components including: 1) User Interest Projection Network (UIPN), which firstly purifies users' behaviors by eliminating the scenario-irrelevant information in behaviors with respect to the visiting scenario, followed by obtaining users' scenario-specific interests by summarizing the purified behaviors with respect to the target item via an attention mechanism; and 2) User Representation Memory Network (URMN), which benefits cold-start users from users with rich behaviors through a memory read and write mechanism. CSMN seamlessly integrates both components in an end-to-end learning framework. Extensive experiments on real-world offline dataset and online A/B test demonstrate the superiority of CSMN over state-of-the-art methods.

cs.IR↗

arXiv2023

Sequence-Based Target Coin Prediction for Cryptocurrency Pump-and-Dump

With the proliferation of pump-and-dump schemes (P&Ds) in the cryptocurrency market, it becomes imperative to detect such fraudulent activities in advance to alert potentially susceptible investors. In this paper, we focus on predicting the pump probability of all coins listed in the target exchange before a scheduled pump time, which we refer to as the target coin prediction task. Firstly, we conduct a comprehensive study of the latest 709 P&D events organized in Telegram from Jan. 2019 to Jan. 2022. Our empirical analysis reveals some interesting patterns of P&Ds, such as that pumped coins exhibit intra-channel homogeneity and inter-channel heterogeneity. Here channel refers a form of group in Telegram that is frequently used to coordinate P&D events. This observation inspires us to develop a novel sequence-based neural network, dubbed SNN, which encodes a channel's P&D event history into a sequence representation via the positional attention mechanism to enhance the prediction accuracy. Positional attention helps to extract useful information and alleviates noise, especially when the sequence length is long. Extensive experiments verify the effectiveness and generalizability of proposed methods. Additionally, we release the code and P&D dataset on GitHub: https://github.com/Bayi-Hu/Pump-and-Dump-Detection-on-Cryptocurrency, and regularly update the dataset.

q-fin.ST↗

arXiv2023

MAMDR: A Model Agnostic Learning Method for Multi-Domain Recommendation

Large-scale e-commercial platforms in the real-world usually contain various recommendation scenarios (domains) to meet demands of diverse customer groups. Multi-Domain Recommendation (MDR), which aims to jointly improve recommendations on all domains and easily scales to thousands of domains, has attracted increasing attention from practitioners and researchers. Existing MDR methods usually employ a shared structure and several specific components to respectively leverage reusable features and domain-specific information. However, data distribution differs across domains, making it challenging to develop a general model that can be applied to all circumstances. Additionally, during training, shared parameters often suffer from the domain conflict while specific parameters are inclined to overfitting on data sparsity domains. we first present a scalable MDR platform served in Taobao that enables to provide services for thousands of domains without specialists involved. To address the problems of MDR methods, we propose a novel model agnostic learning framework, namely MAMDR, for the multi-domain recommendation. Specifically, we first propose a Domain Negotiation (DN) strategy to alleviate the conflict between domains. Then, we develop a Domain Regularization (DR) to improve the generalizability of specific parameters by learning from other domains. We integrate these components into a unified framework and present MAMDR, which can be applied to any model structure to perform multi-domain recommendation. Finally, we present a large-scale implementation of MAMDR in the Taobao application and construct various public MDR benchmark datasets which can be used for following studies. Extensive experiments on both benchmark datasets and industry datasets demonstrate the effectiveness and generalizability of MAMDR.

cs.IR↗

arXiv2023

GIPA: A General Information Propagation Algorithm for Graph Learning

Graph neural networks (GNNs) have been widely used in graph-structured data computation, showing promising performance in various applications such as node classification, link prediction, and network recommendation. Existing works mainly focus on node-wise correlation when doing weighted aggregation of neighboring nodes based on attention, such as dot product by the dense vectors of two nodes. This may cause conflicting noise in nodes to be propagated when doing information propagation. To solve this problem, we propose a General Information Propagation Algorithm (GIPA in short), which exploits more fine-grained information fusion including bit-wise and feature-wise correlations based on edge features in their propagation. Specifically, the bit-wise correlation calculates the element-wise attention weight through a multi-layer perceptron (MLP) based on the dense representations of two nodes and their edge; The feature-wise correlation is based on the one-hot representations of node attribute features for feature selection. We evaluate the performance of GIPA on the Open Graph Benchmark proteins (OGBN-proteins for short) dataset and the Alipay dataset of Alibaba. Experimental results reveal that GIPA outperforms the state-of-the-art models in terms of prediction accuracy, e.g., GIPA achieves an average ROC-AUC of $0.8901\pm 0.0011$, which is better than that of all the existing methods listed in the OGBN-proteins leaderboard.

cs.LG↗

arXiv2023

GraphTheta: A Distributed Graph Neural Network Learning System With Flexible Training Strategy

Graph neural networks (GNNs) have been demonstrated as a powerful tool for analyzing non-Euclidean graph data. However, the lack of efficient distributed graph learning systems severely hinders applications of GNNs, especially when graphs are big and GNNs are relatively deep. Herein, we present GraphTheta, the first distributed and scalable graph learning system built upon vertex-centric distributed graph processing with neural network operators implemented as user-defined functions. This system supports multiple training strategies and enables efficient and scalable big-graph learning on distributed (virtual) machines with low memory. To facilitate graph convolutions, GraphTheta puts forward a new graph learning abstraction named NN-TGAR to bridge the gap between graph processing and graph deep learning. A distributed graph engine is proposed to conduct the stochastic gradient descent optimization with a hybrid-parallel execution, and a new cluster-batched training strategy is supported. We evaluate GraphTheta using several datasets with network sizes ranging from small-, modest- to large-scale. Experimental results show that GraphTheta can scale well to 1,024 workers for training an in-house developed GNN on an industry-scale Alipay dataset of 1.4 billion nodes and 4.1 billion attributed edges, with a cluster of CPU virtual machines (dockers) of small memory each (5$\sim$12GB). Moreover, GraphTheta can outperform DistDGL by up to $2.02\times$, with better scalability, and GraphLearn by up to $30.56\times$. As for model accuracy, GraphTheta is capable of learning as good GNNs as existing frameworks. To the best of our knowledge, this work presents the largest edge-attributed GNN learning task in the literature.

cs.LG↗

arXiv2022

Testing parity symmetry of gravity with gravitational waves

The examination of parity symmetry in gravitational interactions has drawn increasing attention. Although Einstein's General Relativity is parity-conserved, numerous theories of parity-violating (PV) gravity in different frameworks have recently been proposed for different motivations. In this review, we briefly summarize the recent progress of these theories, and focus on the observable effects of PV terms in the gravitational waves (GWs), which are mainly reflected in the difference between the left-hand and right-hand polarization modes. We are primarily concerned with the implications of these theories for GWs generated by the compact binary coalescences and the primordial GWs generated in the early Universe. The deviation of GW waveforms and/or primordial power spectrum can always be quantified by the energy scale of parity violation of the theory. Applying the current and future GW observation from laser interferometers and cosmic microwave background radiation, the current and potential constraints on the PV energy scales are presented, which indicates that the parity symmetry of gravity can be tested in high energy scale in this new era of gravitational waves.

gr-qc↗

arXiv2022

Complete two-loop QCD amplitudes for $tW$ production at hadron colliders

We calculate the complete two-loop QCD amplitudes for hadronic $tW$ production by combining analytical and numerical techniques. The amplitudes have been first reduced to master integrals of eight planar and seven non-planar families, which can contain at most four massive propagators. Then a rational transformation of the master integrals is found to obtain a good basis so that the dimensional parameter decouples from the kinematic variables in the denominators of reduction coefficients. The master integrals are computed by solving their differential equations numerically. We find that the finite part of the two-loop squared amplitude is stable in the bulk of the phase space. After phase space integration and convolution with the parton distributions, it increases the LO cross section by about $3\%$.

hep-ph↗

arXiv2022

Context-Aware Robust Fine-Tuning

Contrastive Language-Image Pre-trained (CLIP) models have zero-shot ability of classifying an image belonging to "[CLASS]" by using similarity between the image and the prompt sentence "a [CONTEXT] of [CLASS]". Based on exhaustive text cues in "[CONTEXT]", CLIP model is aware of different contexts, e.g. background, style, viewpoint, and exhibits unprecedented robustness against a wide range of distribution shifts. However, recent works find further fine-tuning of CLIP models improves accuracy but sacrifices the robustness on downstream tasks. We conduct an empirical investigation to show fine-tuning will corrupt the context-aware ability of pre-trained CLIP features. To solve this problem, we propose Context-Aware Robust Fine-tuning (CAR-FT). CAR-FT regularizes the model during fine-tuning to capture the context information. Specifically, we use zero-shot prompt weights to get the context distribution contained in the image. By minimizing the Kullback-Leibler Divergence (KLD) between context distributions induced by original/fine-tuned CLIP models, CAR-FT makes the context-aware ability of CLIP inherited into downstream tasks, and achieves both higher In-Distribution (ID) and Out-Of-Distribution (OOD) accuracy. The experimental results show CAR-FT achieves superior robustness on five OOD test datasets of ImageNet, and meanwhile brings accuracy gains on nine downstream tasks. Additionally, CAR-FT surpasses previous Domain Generalization (DG) methods and gets 78.5% averaged accuracy on DomainBed benchmark, building the new state-of-the-art.

cs.CV↗

arXiv2022

Towards Graph Self-Supervised Learning with Contrastive Adjusted Zooming

Graph representation learning (GRL) is critical for graph-structured data analysis. However, most of the existing graph neural networks (GNNs) heavily rely on labeling information, which is normally expensive to obtain in the real world. Although some existing works aim to effectively learn graph representations in an unsupervised manner, they suffer from certain limitations, such as the heavy reliance on monotone contrastiveness and limited scalability. To overcome the aforementioned problems, we introduce a novel self-supervised graph representation learning algorithm via Graph Contrastive Adjusted Zooming, namely G-Zoom, to learn node representations by leveraging the proposed adjusted zooming scheme. Specifically, this mechanism enables G-Zoom to explore and extract self-supervision signals from a graph from multiple scales: micro (i.e., node-level), meso (i.e., neighborhood-level), and macro (i.e., subgraph-level). Firstly, we generate two augmented views of the input graph via two different graph augmentations. Then, we establish three different contrastiveness on the above three scales progressively, from node, neighboring, to subgraph level, where we maximize the agreement between graph representations across scales. While we can extract valuable clues from a given graph on the micro and macro perspectives, the neighboring-level contrastiveness offers G-Zoom the capability of a customizable option based on our adjusted zooming scheme to manually choose an optimal viewpoint that lies between the micro and macro perspectives to better understand the graph data. Additionally, to make our model scalable to large graphs, we employ a parallel graph diffusion approach to decouple model training from the graph size. We have conducted extensive experiments on real-world datasets, and the results demonstrate that our proposed model outperforms state-of-the-art methods consistently.

cs.LG↗

arXiv2022

Path-aware Siamese Graph Neural Network for Link Prediction

In this paper, we propose a Path-aware Siamese Graph neural network(PSG) for link prediction tasks. First, PSG captures both nodes and edge features for given two nodes, namely the structure information of k-neighborhoods and relay paths information of the nodes. Furthermore, a novel multi-task GNN framework with self-supervised contrastive learning is proposed for differentiation of positive links and negative links while content and behavior of nodes can be captured simultaneously. We evaluate the proposed algorithm PSG on two link property prediction datasets, ogbl-ddi and ogbl-collab. PSG achieves top 1 performance on ogbl-ddi until submission and top 3 performance on ogbl-collab. The experimental results verify the superiority of our proposed PSG

cs.LG↗

arXiv2022

Carbon Monitor Europe, near-real-time daily CO$_2$ emissions for 27 EU countries and the United Kingdom

With the urgent need to implement the EU countries pledges and to monitor the effectiveness of Green Deal plan, Monitoring Reporting and Verification tools are needed to track how emissions are changing for all the sectors. Current official inventories only provide annual estimates of national CO$_2$ emissions with a lag of 1+ year which do not capture the variations of emissions due to recent shocks including COVID lockdowns and economic rebounds, war in Ukraine. Here we present a near-real-time country-level dataset of daily fossil fuel and cement emissions from January 2019 through December 2021 for 27 EU countries and UK, which called Carbon Monitor Europe. The data are calculated separately for six sectors: power, industry, ground transportation, domestic aviation, international aviation and residential. Daily CO$_2$ emissions are estimated from a large set of activity data compiled from different sources. The goal of this dataset is to improve the timeliness and temporal resolution of emissions for European countries, to inform the public and decision makers about current emissions changes in Europe.

physics.geo-ph↗

arXiv2022

Gravitational Faraday rotation of gravitational waves by a Kerr black hole

Gravitational Faraday Rotation (GFR) is a frame-dragging effect induced by rotating massive objects, which is one of the important, yet studied characteristics of lensed gravitational waves (GWs). In this work, we calculate the GFR angle $χ_g$ of GWs in the weak deflection limit, assuming it is lensed by a Kerr black hole (BH). We find that the GFR effect changes the initial polarization state of the lensed GW. Compared with the Einstein deflection angle, the dominant term of the rotation angle $χ_g$ is a second-order correction to the polarization angle, which depends on the light-of-sight component of BH angular momentum. Such a rotation is tiny and degenerates with the initial polarization angle. In some critical cases, the GFR angle is close to the detection capability of the third-generation GW detector network, although the degeneracy has to be broken.

gr-qc↗

arXiv2022

Oscillatory cooperation prevalence emerges from misperception

Oscillatory behaviors are ubiquitous in nature and the human society. However, most previous works fail to reproduce them in the two-strategy game-theoretical models. Here we show that oscillatory behaviors naturally emerge if incomplete information is incorporated into the cooperation evolution of a non-Markov model. Specifically, we consider a population playing prisoner's dilemma game, where each individual can only probabilistically get access to their neighbors' payoff information and store them within their memory with a given length. They make their decisions based upon these memories. Interestingly, we find that the level of cooperation generally cannot stabilize but render quasi-periodic oscillation, and this observation is strengthened for a longer memory and a smaller information acquisition probability. The mechanism uncovered shows that there are misperceived payoffs about the player's neighborhood, facilitating the growth of cooperators and defectors at different stages that leads to oscillatory behaviors as a result. Our findings are robust to the underlying structure of the population. Given the omnipresence of incomplete information, our findings may provide a plausible explanation for the phenomenon of oscillatory behaviors in the real world.

q-bio.PE↗

arXiv2022

Complete two-loop electroweak corrections to $e^+e^-\rightarrow HZ$

We compute the complete two-loop electroweak corrections to the Higgsstralung process $e^+e^-\rightarrow HZ$ at the future Higgs factory. The Feynman integrals involved in the computation are decomposed into linear combinations of a minimal set of master integrals taking advantage of the recent developments of integral reduction techniques. The master integrals are then evaluated by differential equations with boundary conditions provided by the auxiliary mass flow method. Our final result for given $\sqrt{s}$ is expressed as a piecewise function defined by several deeply expanded power series, which has high precision and can be further manipulated efficiently. Our calculation presents the first complete two-loop electroweak corrections for processes with four external particles.

hep-ph↗

arXiv2022

GIFT: Graph-guIded Feature Transfer for Cold-Start Video Click-Through Rate Prediction

Short video has witnessed rapid growth in the past few years in e-commerce platforms like Taobao. To ensure the freshness of the content, platforms need to release a large number of new videos every day, making conventional click-through rate (CTR) prediction methods suffer from the item cold-start problem. In this paper, we propose GIFT, an efficient Graph-guIded Feature Transfer system, to fully take advantages of the rich information of warmed-up videos to compensate for the cold-start ones. Specifically, we establish a heterogeneous graph that contains physical and semantic linkages to guide the feature transfer process from warmed-up video to cold-start videos. The physical linkages represent explicit relationships, while the semantic linkages measure the proximity of multi-modal representations of two videos. We elaborately design the feature transfer function to make aware of different types of transferred features (e.g., id representations and historical statistics) from different metapaths on the graph. We conduct extensive experiments on a large real-world dataset, and the results show that our GIFT system outperforms SOTA methods significantly and brings a 6.82% lift on CTR in the homepage of Taobao App.

cs.IR↗