Searcharxiv⌕ Search

arXiv subjects

Xi Yin

Publications and source records attributed to Xi Yin.

At least 55 records · Page 3Linked to original sources
arXiv2022

The S-Matrix of 2D Type 0B String Theory Part 1: Perturbation Theory Revisited

We study the perturbative S-matrix of closed strings in the two-dimensional type 0B string theory from the worldsheet perspective, by directly integrating correlation functions of ${\cal N}=1$ Liouville theory. The latter is computed numerically using recurrence relations for super-Virasoro conformal blocks. We show that the tree level 3- and 4-point amplitudes are in agreement with the proposed dual matrix quantum mechanics. The non-perturbative aspects of the duality will be analyzed in a companion paper.

hep-th↗
arXiv2022

The S-Matrix of 2D Type 0B String Theory Part 2: D-Instanton Effects

We study the effect of D-instantons on closed string scattering amplitudes in the two-dimensional type 0B string theory from the worldsheet perspective. We find that the contribution from a pair of D-instanton and anti-D-instanton to the closed string reflection amplitude, with a suitable contour prescription for the integration over the D-instanton moduli space, agrees with the corresponding leading non-perturbative corrections in the proposed dual matrix quantum mechanics. This analysis is further extended to thermal observables defined at finite temperature. The infrared divergence in charged processes is understood through the measure factor for charged D-instantons, and can be treated with spacetime dimensional regularization.

hep-th↗
arXiv2022

Proactive Image Manipulation Detection

Image manipulation detection algorithms are often trained to discriminate between images manipulated with particular Generative Models (GMs) and genuine/real images, yet generalize poorly to images manipulated with GMs unseen in the training. Conventional detection algorithms receive an input image passively. By contrast, we propose a proactive scheme to image manipulation detection. Our key enabling technique is to estimate a set of templates which when added onto the real image would lead to more accurate manipulation detection. That is, a template protected real image, and its manipulated version, is better discriminated compared to the original real image vs. its manipulated one. These templates are estimated using certain constraints based on the desired properties of templates. For image manipulation detection, our proposed approach outperforms the prior work by an average precision of 16% for CycleGAN and 32% for GauGAN. Our approach is generalizable to a variety of GMs showing an improvement over prior work by an average precision of 10% averaged across 12 GMs. Our code is available at https://www.github.com/vishal3477/proactive_IMD.

cs.CV↗
arXiv2021

img2pose: Face Alignment and Detection via 6DoF, Face Pose Estimation

We propose real-time, six degrees of freedom (6DoF), 3D face pose estimation without face detection or landmark localization. We observe that estimating the 6DoF rigid transformation of a face is a simpler problem than facial landmark detection, often used for 3D face alignment. In addition, 6DoF offers more information than face bounding box labels. We leverage these observations to make multiple contributions: (a) We describe an easily trained, efficient, Faster R-CNN--based model which regresses 6DoF pose for all faces in the photo, without preliminary face detection. (b) We explain how pose is converted and kept consistent between the input photo and arbitrary crops created while training and evaluating our model. (c) Finally, we show how face poses can replace detection bounding box training labels. Tests on AFLW2000-3D and BIWI show that our method runs at real-time and outperforms state of the art (SotA) face pose estimators. Remarkably, our method also surpasses SotA models of comparable complexity on the WIDER FACE detection benchmark, despite not been optimized on bounding box labels.

cs.CV↗
arXiv2021

KGSynNet: A Novel Entity Synonyms Discovery Framework with Knowledge Graph

Entity synonyms discovery is crucial for entity-leveraging applications. However, existing studies suffer from several critical issues: (1) the input mentions may be out-of-vocabulary (OOV) and may come from a different semantic space of the entities; (2) the connection between mentions and entities may be hidden and cannot be established by surface matching; and (3) some entities rarely appear due to the long-tail effect. To tackle these challenges, we facilitate knowledge graphs and propose a novel entity synonyms discovery framework, named \emph{KGSynNet}. Specifically, we pre-train subword embeddings for mentions and entities using a large-scale domain-specific corpus while learning the knowledge embeddings of entities via a joint TransC-TransE model. More importantly, to obtain a comprehensive representation of entities, we employ a specifically designed \emph{fusion gate} to adaptively absorb the entities' knowledge information into their semantic features. We conduct extensive experiments to demonstrate the effectiveness of our \emph{KGSynNet} in leveraging the knowledge graph. The experimental results show that the \emph{KGSynNet} improves the state-of-the-art methods by 14.7\% in terms of hits@3 in the offline evaluation and outperforms the BERT model by 8.3\% in the positive feedback rate of an online A/B test on the entity linking module of a question answering system.

cs.AI↗
arXiv2021

A Multiplexed Network for End-to-End, Multilingual OCR

Recent advances in OCR have shown that an end-to-end (E2E) training pipeline that includes both detection and recognition leads to the best results. However, many existing methods focus primarily on Latin-alphabet languages, often even only case-insensitive English characters. In this paper, we propose an E2E approach, Multiplexed Multilingual Mask TextSpotter, that performs script identification at the word level and handles different scripts with different recognition heads, all while maintaining a unified loss that simultaneously optimizes script identification and multiple recognition heads. Experiments show that our method outperforms the single-head model with similar number of parameters in end-to-end recognition tasks, and achieves state-of-the-art results on MLT17 and MLT19 joint text detection and script identification benchmarks. We believe that our work is a step towards the end-to-end trainable and scalable multilingual multi-purpose OCR system. Our code and model will be released.

cs.CV↗
arXiv2021

VIVO: Visual Vocabulary Pre-Training for Novel Object Captioning

It is highly desirable yet challenging to generate image captions that can describe novel objects which are unseen in caption-labeled training data, a capability that is evaluated in the novel object captioning challenge (nocaps). In this challenge, no additional image-caption training data, other thanCOCO Captions, is allowed for model training. Thus, conventional Vision-Language Pre-training (VLP) methods cannot be applied. This paper presents VIsual VOcabulary pretraining (VIVO) that performs pre-training in the absence of caption annotations. By breaking the dependency of paired image-caption training data in VLP, VIVO can leverage large amounts of paired image-tag data to learn a visual vocabulary. This is done by pre-training a multi-layer Transformer model that learns to align image-level tags with their corresponding image region features. To address the unordered nature of image tags, VIVO uses a Hungarian matching loss with masked tag prediction to conduct pre-training. We validate the effectiveness of VIVO by fine-tuning the pre-trained model for image captioning. In addition, we perform an analysis of the visual-text alignment inferred by our model. The results show that our model can not only generate fluent image captions that describe novel objects, but also identify the locations of these objects. Our single model has achieved new state-of-the-art results on nocaps and surpassed the human CIDEr score.

cs.CV↗
arXiv2020

TAP: Text-Aware Pre-training for Text-VQA and Text-Caption

In this paper, we propose Text-Aware Pre-training (TAP) for Text-VQA and Text-Caption tasks. These two tasks aim at reading and understanding scene text in images for question answering and image caption generation, respectively. In contrast to the conventional vision-language pre-training that fails to capture scene text and its relationship with the visual and text modalities, TAP explicitly incorporates scene text (generated from OCR engines) in pre-training. With three pre-training tasks, including masked language modeling (MLM), image-text (contrastive) matching (ITM), and relative (spatial) position prediction (RPP), TAP effectively helps the model learn a better aligned representation among the three modalities: text word, visual object, and scene text. Due to this aligned representation learning, even pre-trained on the same downstream task dataset, TAP already boosts the absolute accuracy on the TextVQA dataset by +5.4%, compared with a non-TAP baseline. To further improve the performance, we build a large-scale dataset based on the Conceptual Caption dataset, named OCR-CC, which contains 1.4 million scene text-related image-text pairs. Pre-trained on this OCR-CC dataset, our approach outperforms the state of the art by large margins on multiple tasks, i.e., +8.3% accuracy on TextVQA, +8.6% accuracy on ST-VQA, and +10.2 CIDEr score on TextCaps.

cs.CV↗
arXiv2020

Strings in Ramond-Ramond Backgrounds from the Neveu-Schwarz-Ramond Formalism

We treat RR flux backgrounds of type II string theory in the framework of closed superstring field theory based on the NSR formalism, focusing on two examples: (1) the pp-wave background supported by 5-form flux, and (2) $AdS_3\times S^3\times M_4$ supported by mixed 3-form fluxes. In both cases, we analyze the classical string field solution perturbatively, and compute the correction to the dispersion relation of string states to quadratic order in the RR flux. In the first example, our result is in a delicate way consistent with that obtained from lightcone quantization of the Green-Schwarz string. In the second example, we will obtain numerically the mass corrections to pulsating type IIB strings in $AdS_3\times S^3\times M_4$. Our results, valid at finite $AdS$ radius, agree with previously known answers in the semiclassical limit and in the BMN limit respectively.

hep-th↗
arXiv2020

FAN: Feature Adaptation Network for Surveillance Face Recognition and Normalization

This paper studies face recognition (FR) and normalization in surveillance imagery. Surveillance FR is a challenging problem that has great values in law enforcement. Despite recent progress in conventional FR, less effort has been devoted to surveillance FR. To bridge this gap, we propose a Feature Adaptation Network (FAN) to jointly perform surveillance FR and normalization. Our face normalization mainly acts on the aspect of image resolution, closely related to face super-resolution. However, previous face super-resolution methods require paired training data with pixel-to-pixel correspondence, which is typically unavailable between real-world low-resolution and high-resolution faces. FAN can leverage both paired and unpaired data as we disentangle the features into identity and non-identity components and adapt the distribution of the identity features, which breaks the limit of current face super-resolution methods. We further propose a random scale augmentation scheme to learn resolution robust identity features, with advantages over previous fixed scale augmentation. Extensive experiments on LFW, WIDER FACE, QUML-SurvFace and SCface datasets have shown the effectiveness of our method on surveillance FR and normalization.

cs.CV↗
arXiv2020

Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks

Large-scale pre-training methods of learning cross-modal representations on image-text pairs are becoming popular for vision-language tasks. While existing methods simply concatenate image region features and text features as input to the model to be pre-trained and use self-attention to learn image-text semantic alignments in a brute force manner, in this paper, we propose a new learning method Oscar (Object-Semantics Aligned Pre-training), which uses object tags detected in images as anchor points to significantly ease the learning of alignments. Our method is motivated by the observation that the salient objects in an image can be accurately detected, and are often mentioned in the paired text. We pre-train an Oscar model on the public corpus of 6.5 million text-image pairs, and fine-tune it on downstream tasks, creating new state-of-the-arts on six well-established vision-language understanding and generation tasks.

cs.CV↗
arXiv2020

Hashing-based Non-Maximum Suppression for Crowded Object Detection

In this paper, we propose an algorithm, named hashing-based non-maximum suppression (HNMS) to efficiently suppress the non-maximum boxes for object detection. Non-maximum suppression (NMS) is an essential component to suppress the boxes at closely located locations with similar shapes. The time cost tends to be huge when the number of boxes becomes large, especially for crowded scenes. The basic idea of HNMS is to firstly map each box to a discrete code (hash cell) and then remove the boxes with lower confidences if they are in the same cell. Considering the intersection-over-union (IoU) as the metric, we propose a simple yet effective hashing algorithm, named IoUHash, which guarantees that the boxes within the same cell are close enough by a lower IoU bound. For two-stage detectors, we replace NMS in region proposal network with HNMS, and observe significant speed-up with comparable accuracy. For one-stage detectors, HNMS is used as a pre-filter to speed up the suppression with a large margin. Extensive experiments are conducted on CARPK, SKU-110K, CrowdHuman datasets to demonstrate the efficiency and effectiveness of HNMS. Code is released at \url{https://github.com/microsoft/hnms.git}.

cs.CV↗
arXiv2019

ZZ Instantons and the Non-Perturbative Dual of c = 1 String Theory

We study the effect of ZZ instantons in c = 1 string theory, and demonstrate that they give rise to non-perturbative corrections to scattering amplitudes that do not saturate unitarity within the closed string sector. Beyond the leading non-perturbative order, logarithmic divergences are canceled between worldsheet diagrams of different topologies, due to the Fischler-Susskind-Polchinski mechanism. We propose that the closed string vacuum in c = 1 string theory is non-perturbatively dual to a state of the matrix quantum mechanics in which all scattering states up to a given energy with no incoming flux from the "other side" of the potential are occupied by free fermions. Under such a proposal, we find detailed agreement of non-perturbative corrections to closed string amplitudes in the worldsheet description and in the dual matrix model.

hep-th↗
arXiv2019

Multi-Instanton Calculus in c = 1 String Theory

We formulate a strategy for computing the complete set of non-perturbative corrections to closed string scattering in $c=1$ string theory from the worldsheet perspective. This requires taking into account the effect of multiple ZZ-instantons, including higher instantons constructed from ZZ boundary conditions of type $(m,1)$, with a careful treatment of the measure and contour in the integration over the instanton moduli space. The only a priori ambiguity in our prescription is a normalization constant ${\cal N}_{m}$ that appears in the integration measure for the $(m,1)$-type ZZ instanton, at each positive integer $m$. We investigate leading corrections to the closed string reflection amplitude at the $n$-instanton level, i.e. of order $e^{-n/g_s}$, and find striking agreement with our recent proposal on the non-perturbative completion of the dual matrix quantum mechanics, which in turn fixes ${\cal N}_{m}$ for all $m$.

hep-th↗
arXiv2019

Feature Transfer Learning for Deep Face Recognition with Under-Represented Data

Despite the large volume of face recognition datasets, there is a significant portion of subjects, of which the samples are insufficient and thus under-represented. Ignoring such significant portion results in insufficient training data. Training with under-represented data leads to biased classifiers in conventionally-trained deep networks. In this paper, we propose a center-based feature transfer framework to augment the feature space of under-represented subjects from the regular subjects that have sufficiently diverse samples. A Gaussian prior of the variance is assumed across all subjects and the variance from regular ones are transferred to the under-represented ones. This encourages the under-represented distribution to be closer to the regular distribution. Further, an alternating training regimen is proposed to simultaneously achieve less biased classifiers and a more discriminative feature representation. We conduct ablative study to mimic the under-represented datasets by varying the portion of under-represented classes on the MS-Celeb-1M dataset. Advantageous results on LFW, IJB-A and MS-Celeb-1M demonstrate the effectiveness of our feature transfer and training strategy, compared to both general baselines and state-of-the-art methods. Moreover, our feature transfer successfully presents smooth visual interpolation, which conducts disentanglement to preserve identity of a class while augmenting its feature space with non-identity variations such as pose and lighting.

cs.CV↗
arXiv2019

The M-Theory S-Matrix From ABJM: Beyond 11D Supergravity

We show that by studying the flat spacetime limit of the Mellin amplitude associated with the four-point correlation function of scalar operators in the stress tensor multiplet of ABJM theory, one can produce the momentum expansion of the M-theory four-graviton S-matrix elements. Using CFT data previously obtained from the supersymmetric localization method, we carry out this procedure explicitly to the second nontrivial order in the momentum expansion, and recover precisely the known $R^4$ contribution to the scattering amplitude of super-gravitons in M-theory in eleven dimensions.

hep-th↗