arXiv subjects

Shuai Zhao

Publications and source records attributed to Shuai Zhao.

At least 127 records · Page 7Linked to original sources

arXiv2022

Realigned Hardy's Paradox

Hardy's paradox provides an all-versus-nothing fashion to directly certify that quantum mechanics cannot be completely described by local realistic theory. However, when considering potential imperfections in experiments, like imperfect entanglement source and low detection efficiency, the original Hardy's paradox may induce a rather small Hardy violation and only be realized by expensive quantum systems. To overcome this problem, we propose a realigned Hardy's paradox. Compared with the original version of Hardy's paradox, the realigned Hardy's paradox can dramatically improve the Hardy violation. Then, we generalize the realigned Hardy's paradox to arbitrary even $n$ dichotomic measurements. For $n=2$ and $n=4$ cases, the realigned Hardy's paradox can achieve Hardy values $P(00|A_1B_1)$ approximate $0.4140$ and $0.7734$ respectively compared with $0.09$ of the original Hardy's paradox. Meanwhile, the structure of the realigned Hardy's paradox is simpler and more robust in the sense that there is only one Hardy condition rather than three conditions. One can anticipate that the realigned Hardy's paradox can tolerate more experimental imperfections and stimulate more fascinating quantum information applications.

quant-ph↗

arXiv2022

A generic framework for genuine multipartite entanglement detection

Design of detection strategies for multipartite entanglement stands as a central importance on our understanding of fundamental quantum mechanics and has had substantial impact on quantum information applications. However, accurate and robust detection approaches are severely hindered, particularly when the number of nodes grows rapidly like in a quantum network. Here we present an exquisite procedure that generates novel entanglement witness for arbitrary targeted state via a generic and operational framework. The framework enjoys a systematic and high-efficient character and allows to substantiate genuine multipartite entanglement for a variety of states that arise naturally in practical situations, and to dramatically outperform currently standard methods. With excellent noise tolerance, our framework should be broadly applicable to witness genuine multipartite entanglement in various practically scenarios, and to facilitate making the best use of entangled resources in the emerging area of quantum network.

quant-ph↗

arXiv2022

Generative Prompt Tuning for Relation Classification

Using prompts to explore the knowledge contained within pre-trained language models for downstream tasks has now become an active topic. Current prompt tuning methods mostly convert the downstream tasks to masked language modeling problems by adding cloze-style phrases and mapping all labels to verbalizations with fixed length, which has proven effective for tasks with simple label spaces. However, when applied to relation classification exhibiting complex label spaces, vanilla prompt tuning methods may struggle with label verbalizations with arbitrary lengths due to rigid prompt restrictions. Inspired by the text infilling task for pre-training generative models that can flexibly predict missing spans, we propose a novel generative prompt tuning method to reformulate relation classification as an infilling problem, which frees our approach from limitations of current prompt based approaches and thus fully exploits rich semantics of entity and relation types. In addition, we design entity-guided decoding and discriminative relation scoring to generate and align relations effectively and efficiently during inference. Extensive experiments under fully supervised settings and low-resource settings demonstrate the effectiveness of our approach.

cs.CL↗

arXiv2022

Perturbative unitarity and NEC violation in genesis cosmology

Explorations of the violation of null energy condition (NEC) in cosmology could enrich our understanding of the very early universe and the related gravity theories. Although a fully stable NEC violation can be realized in the ``beyond Horndeski'' theory, it remains an open question whether a violation of the NEC is allowed by some fundamental properties of UV-complete theories or the consistency requirements of effective field theory (EFT). We investigate the tree-level perturbative unitarity for stable NEC violations in the contexts of both Galileon and ``beyond Horndeski'' genesis cosmology, in which the universe is asymptotically Minkowskian in the past. We find that the constraints of perturbative unitarity imply that we may need some unknown new physics below the cut-off scale of the EFT other than that represented by the ``beyond Horndeski'' operators.

gr-qc↗

arXiv2022

Towards Better Accuracy-efficiency Trade-offs: Divide and Co-training

The width of a neural network matters since increasing the width will necessarily increase the model capacity. However, the performance of a network does not improve linearly with the width and soon gets saturated. In this case, we argue that increasing the number of networks (ensemble) can achieve better accuracy-efficiency trade-offs than purely increasing the width. To prove it, one large network is divided into several small ones regarding its parameters and regularization components. Each of these small networks has a fraction of the original one's parameters. We then train these small networks together and make them see various views of the same data to increase their diversity. During this co-training process, networks can also learn from each other. As a result, small networks can achieve better ensemble performance than the large one with few or no extra parameters or FLOPs, \ie, achieving better accuracy-efficiency trade-offs. Small networks can also achieve faster inference speed than the large one by concurrent running. All of the above shows that the number of networks is a new dimension of model scaling. We validate our argument with 8 different neural architectures on common benchmarks through extensive experiments. The code is available at \url{https://github.com/FreeformRobotics/Divide-and-Co-training}.

cs.CV↗

arXiv2022

Latent Heterogeneous Graph Network for Incomplete Multi-View Learning

Multi-view learning has progressed rapidly in recent years. Although many previous studies assume that each instance appears in all views, it is common in real-world applications for instances to be missing from some views, resulting in incomplete multi-view data. To tackle this problem, we propose a novel Latent Heterogeneous Graph Network (LHGN) for incomplete multi-view learning, which aims to use multiple incomplete views as fully as possible in a flexible manner. By learning a unified latent representation, a trade-off between consistency and complementarity among different views is implicitly realized. To explore the complex relationship between samples and latent representations, a neighborhood constraint and a view-existence constraint are proposed, for the first time, to construct a heterogeneous graph. Finally, to avoid any inconsistencies between training and test phase, a transductive learning technique is applied based on graph learning for classification tasks. Extensive experimental results on real-world datasets demonstrate the effectiveness of our model over existing state-of-the-art approaches.

cs.LG↗

arXiv2022

Learning Personalized Representations using Graph Convolutional Network

Generating representations that precisely reflect customers' behavior is an important task for providing personalized skill routing experience in Alexa. Currently, Dynamic Routing (DR) team, which is responsible for routing Alexa traffic to providers or skills, relies on two features to be served as personal signals: absolute traffic count and normalized traffic count of every skill usage per customer. Neither of them considers the network based structure for interactions between customers and skills, which contain richer information for customer preferences. In this work, we first build a heterogeneous edge attributed graph based customers' past interactions with the invoked skills, in which the user requests (utterances) are modeled as edges. Then we propose a graph convolutional network(GCN) based model, namely Personalized Dynamic Routing Feature Encoder(PDRFE), that generates personalized customer representations learned from the built graph. Compared with existing models, PDRFE is able to further capture contextual information in the graph convolutional function. The performance of our proposed model is evaluated by a downstream task, defect prediction, that predicts the defect label from the learned embeddings of customers and their triggered skills. We observe up to 41% improvements on the cross entropy metric for our proposed models compared to the baselines.

cs.LG↗

arXiv2022

Inverse moment of the $B$-meson quasi distribution amplitude

We perform a study on the structure of inverse moment (IM) of quasi distributions, by taking $B$-meson quasi distribution amplitude (quasi-DA) as an example. Based on a one-loop calculation, we derive the renormalization group equation and velocity evolution equation for the first IM of quasi-DA. We find that, in the large velocity limit, the first IM of $B$-meson quasi-DA can be factorized into IM as well as logarithmic moments of light-cone distribution amplitude (LCDA), accompanied by short distance coefficients. Our results can be useful either in understanding the patterns of perturbative matching in Large Momentum Effective Theory or evaluating inverse moment of $B$-meson LCDA on the lattice.

hep-ph↗

arXiv2022

Analyzing Modality Robustness in Multimodal Sentiment Analysis

Building robust multimodal models are crucial for achieving reliable deployment in the wild. Despite its importance, less attention has been paid to identifying and improving the robustness of Multimodal Sentiment Analysis (MSA) models. In this work, we hope to address that by (i) Proposing simple diagnostic checks for modality robustness in a trained multimodal model. Using these checks, we find MSA models to be highly sensitive to a single modality, which creates issues in their robustness; (ii) We analyze well-known robust training strategies to alleviate the issues. Critically, we observe that robustness can be achieved without compromising on the original performance. We hope our extensive study-performed across five models and two benchmark datasets-and proposed procedures would make robustness an integral component in MSA research. Our diagnostic checks and robust training solutions are simple to implement and available at https://github. com/declare-lab/MSA-Robustness.

cs.CL↗

arXiv2022

Exploring Entity Interactions for Few-Shot Relation Learning (Student Abstract)

Few-shot relation learning refers to infer facts for relations with a limited number of observed triples. Existing metric-learning methods for this problem mostly neglect entity interactions within and between triples. In this paper, we explore this kind of fine-grained semantic meanings and propose our model TransAM. Specifically, we serialize reference entities and query entities into sequence and apply transformer structure with local-global attention to capture both intra- and inter-triple entity interactions. Experiments on two public benchmark datasets NELL-One and Wiki-One with 1-shot setting prove the effectiveness of TransAM.

cs.CL↗

arXiv2022

CenterCLIP: Token Clustering for Efficient Text-Video Retrieval

Recently, large-scale pre-training methods like CLIP have made great progress in multi-modal research such as text-video retrieval. In CLIP, transformers are vital for modeling complex multi-modal relations. However, in the vision transformer of CLIP, the essential visual tokenization process, which produces discrete visual token sequences, generates many homogeneous tokens due to the redundancy nature of consecutive and similar frames in videos. This significantly increases computation costs and hinders the deployment of video retrieval models in web applications. In this paper, to reduce the number of redundant video tokens, we design a multi-segment token clustering algorithm to find the most representative tokens and drop the non-essential ones. As the frame redundancy occurs mostly in consecutive frames, we divide videos into multiple segments and conduct segment-level clustering. Center tokens from each segment are later concatenated into a new sequence, while their original spatial-temporal relations are well maintained. We instantiate two clustering algorithms to efficiently find deterministic medoids and iteratively partition groups in high dimensional space. Through this token clustering and center selection procedure, we successfully reduce computation costs by removing redundant visual tokens. This method further enhances segment-level semantic alignment between video and text representations, enforcing the spatio-temporal interactions of tokens from within-segment frames. Our method, coined as CenterCLIP, surpasses existing state-of-the-art by a large margin on typical text-video benchmarks, while reducing the training memory cost by 35\% and accelerating the inference speed by 14\% at the best case. The code is available at \href{https://github.com/mzhaoshuai/CenterCLIP}{https://github.com/mzhaoshuai/CenterCLIP}.

cs.CV↗

arXiv2022

WuDaoMM: A large-scale Multi-Modal Dataset for Pre-training models

Compared with the domain-specific model, the vision-language pre-training models (VLPMs) have shown superior performance on downstream tasks with fast fine-tuning process. For example, ERNIE-ViL, Oscar and UNIMO trained VLPMs with a uniform transformers stack architecture and large amounts of image-text paired data, achieving remarkable results on downstream tasks such as image-text reference(IR and TR), vision question answering (VQA) and image captioning (IC) etc. During the training phase, VLPMs are always fed with a combination of multiple public datasets to meet the demand of large-scare training data. However, due to the unevenness of data distribution including size, task type and quality, using the mixture of multiple datasets for model training can be problematic. In this work, we introduce a large-scale multi-modal corpora named WuDaoMM, totally containing more than 650M image-text pairs. Specifically, about 600 million pairs of data are collected from multiple webpages in which image and caption present weak correlation, and the other 50 million strong-related image-text pairs are collected from some high-quality graphic websites. We also release a base version of WuDaoMM with 5 million strong-correlated image-text pairs, which is sufficient to support the common cross-modal model pre-training. Besides, we trained both an understanding and a generation vision-language (VL) model to test the dataset effectiveness. The results show that WuDaoMM can be applied as an efficient dataset for VLPMs, especially for the model in text-to-image generation task. The data is released at https://data.wudaoai.cn

cs.CV↗

arXiv2022

SCALoss: Side and Corner Aligned Loss for Bounding Box Regression

Bounding box regression is an important component in object detection. Recent work achieves promising performance by optimizing the Intersection over Union~(IoU). However, IoU-based loss has the gradient vanish problem in the case of low overlapping bounding boxes, and the model could easily ignore these simple cases. In this paper, we propose Side Overlap~(SO) loss by maximizing the side overlap of two bounding boxes, which puts more penalty for low overlapping bounding box cases. Besides, to speed up the convergence, the Corner Distance~(CD) is added into the objective function. Combining the Side Overlap and Corner Distance, we get a new regression objective function, \textit{Side and Corner Align Loss~(SCALoss)}. The SCALoss is well-correlated with IoU loss, which also benefits the evaluation metric but produces more penalty for low-overlapping cases. It can serve as a comprehensive similarity measure, leading to better localization performance and faster convergence speed. Experiments on COCO, PASCAL VOC, and LVIS benchmarks show that SCALoss can bring consistent improvement and outperform $\ell_n$ loss and IoU based loss with popular object detectors such as YOLOV3, SSD, Faster-RCNN. Code is available at: \url{https://github.com/Turoad/SCALoss}.

cs.CV↗

arXiv2022

A Roadmap for Big Model

With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm. Researchers have achieved various outcomes in the construction of BMs and the BM application in many fields. At present, there is a lack of research work that sorts out the overall progress of BMs and guides the follow-up research. In this paper, we cover not only the BM technologies themselves but also the prerequisites for BM training and applications with BMs, dividing the BM review into four parts: Resource, Models, Key Technologies and Application. We introduce 16 specific BM-related topics in those four parts, they are Data, Knowledge, Computing System, Parallel Training System, Language Model, Vision Model, Multi-modal Model, Theory&Interpretability, Commonsense Reasoning, Reliability&Security, Governance, Evaluation, Machine Translation, Text Generation, Dialogue and Protein Research. In each topic, we summarize clearly the current studies and propose some future research directions. At the end of this paper, we conclude the further development of BMs in a more general view.

cs.LG↗

arXiv2022

Parton distribution function for topological charge at one loop

We present results for the $gg$-part of the one-loop corrections to the recently introduced "topological charge" GPD $\widetilde F(x,q^2)$. In particular, we give expression for its evolution kernel. To enforce strict compliance with the gauge invariance requirements, we have used on-shell states for external gluons, and have obtained identical results both in Feynman and light-cone gauges. No "zero mode" $δ(x)$ terms were found for the twist-4 gluon GPD $\widetilde F(x,q^2)$.

hep-ph↗

arXiv2021

Robust self-testing of multipartite GHZ-state measurements in quantum networks

Self-testing is a device-independent examination of quantum devices based on correlations of observed statistics. Motivated by elegant progresses on self-testing strategies for measurements [Phys. Rev. Lett. 121, 250507 (2018)] and for states [New J. Phys. 20, 083041 (2018)], we develop a general self-testing procedure for multipartite generalized GHZ-state measurements. The key step is self-testing all measurement eigenstates for a general N-qubit multipartite GHZ-state measurement. Following our procedure, one only needs to perform local measurements on N-2 chosen parties of the N-partite eigenstate and maintain to achieve the maximal violation of tilted Clauser-Horne-Shimony-Holt (CHSH) Bell inequality for remaining two parties. Moreover, this approach is physically operational from an experimental point of view. It turns out that the existing result for three-qubit GHZ-state measurement is recovered as a special case. Meanwhile, we develop the self-testing method to be robust against certain white noise.

quant-ph↗

arXiv2021

Renormalization and Mixing of Staple-Shaped Wilson Line Operators on the Lattice Revisited

Transverse-momentum-dependent parton distribution functions and wave functions (TMDPDFs/TMDWFs) can be extracted from lattice calculations of appropriate Euclidean matrix elements of staple-shaped Wilson line operators. We investigate the mixing pattern of such operators under lattice renormalization using symmetry considerations. We perform an analysis for operators with all Dirac structures, which reveals mixings that are not present in one-loop lattice perturbation theory calculations. We also present the relevant one-loop matching in a renormalization scheme that does not introduce extra non-perturbative effects at large distances, both for the TMDPDFs and for the TMDWFs. Our results have the potential to greatly facilitate numerical calculations of TMDPDFs and TMDWFs on the lattice.

hep-ph↗

arXiv2021

Learning Dynamic Compact Memory Embedding for Deformable Visual Object Tracking

Recently, template-based trackers have become the leading tracking algorithms with promising performance in terms of efficiency and accuracy. However, the correlation operation between query feature and the given template only exploits accurate target localization, leading to state estimation error especially when the target suffers from severe deformable variations. To address this issue, segmentation-based trackers have been proposed that employ per-pixel matching to improve the tracking performance of deformable objects effectively. However, most of existing trackers only refer to the target features in the initial frame, thereby lacking the discriminative capacity to handle challenging factors, e.g., similar distractors, background clutter, appearance change, etc. To this end, we propose a dynamic compact memory embedding to enhance the discrimination of the segmentation-based deformable visual tracking method. Specifically, we initialize a memory embedding with the target features in the first frame. During the tracking process, the current target features that have high correlation with existing memory are updated to the memory embedding online. To further improve the segmentation accuracy for deformable objects, we employ a point-to-global matching strategy to measure the correlation between the pixel-wise query features and the whole template, so as to capture more detailed deformation information. Extensive evaluations on six challenging tracking benchmarks including VOT2016, VOT2018, VOT2019, GOT-10K, TrackingNet, and LaSOT demonstrate the superiority of our method over recent remarkable trackers. Besides, our method outperforms the excellent segmentation-based trackers, i.e., D3S and SiamMask on DAVIS2017 benchmark.

cs.CV↗