arXiv subjects

Zejun Li

Publications and source records attributed to Zejun Li.

25 records · Page 2Linked to original sources

arXiv2021

Constructing Phrase-level Semantic Labels to Form Multi-Grained Supervision for Image-Text Retrieval

Existing research for image text retrieval mainly relies on sentence-level supervision to distinguish matched and mismatched sentences for a query image. However, semantic mismatch between an image and sentences usually happens in finer grain, i.e., phrase level. In this paper, we explore to introduce additional phrase-level supervision for the better identification of mismatched units in the text. In practice, multi-grained semantic labels are automatically constructed for a query image in both sentence-level and phrase-level. We construct text scene graphs for the matched sentences and extract entities and triples as the phrase-level labels. In order to integrate both supervision of sentence-level and phrase-level, we propose Semantic Structure Aware Multimodal Transformer (SSAMT) for multi-modal representation learning. Inside the SSAMT, we utilize different kinds of attention mechanisms to enforce interactions of multi-grain semantic units in both sides of vision and language. For the training, we propose multi-scale matching losses from both global and local perspectives, and penalize mismatched phrases. Experimental results on MS-COCO and Flickr30K show the effectiveness of our approach compared to some state-of-the-art models.

cs.CV↗

arXiv2021

TCIC: Theme Concepts Learning Cross Language and Vision for Image Captioning

Existing research for image captioning usually represents an image using a scene graph with low-level facts (objects and relations) and fails to capture the high-level semantics. In this paper, we propose a Theme Concepts extended Image Captioning (TCIC) framework that incorporates theme concepts to represent high-level cross-modality semantics. In practice, we model theme concepts as memory vectors and propose Transformer with Theme Nodes (TTN) to incorporate those vectors for image captioning. Considering that theme concepts can be learned from both images and captions, we propose two settings for their representations learning based on TTN. On the vision side, TTN is configured to take both scene graph based features and theme concepts as input for visual representation learning. On the language side, TTN is configured to take both captions and theme concepts as input for text representation re-construction. Both settings aim to generate target captions with the same transformer-based decoder. During the training, we further align representations of theme concepts learned from images and corresponding captions to enforce the cross-modality learning. Experimental results on MS COCO show the effectiveness of our approach compared to some state-of-the-art models.

cs.CV↗

arXiv2021

Ferromagnetism in 2D Vanadium Diselenide

Two-dimensional (2D) Van der Waals ferromagnets carry the promise of ultimately miniature spintronics and information storage devices. Among the newly discovered 2D ferromagnets all inherit the magnetic ordering from their bulk ancestors. Here we report a new 2D ferromagnetic semiconductor at room temperature, 2H phase vanadium diselenide (VSe2) which show ferromagnetic at 2D form only. This unique 2D ferromagnetic semiconductor manifests an enhanced magnetic ordering owing to structural anisotropy at 2D limit.

cond-mat.mtrl-sci↗

arXiv2021

An Unsupervised Sampling Approach for Image-Sentence Matching Using Document-Level Structural Information

In this paper, we focus on the problem of unsupervised image-sentence matching. Existing research explores to utilize document-level structural information to sample positive and negative instances for model training. Although the approach achieves positive results, it introduces a sampling bias and fails to distinguish instances with high semantic similarity. To alleviate the bias, we propose a new sampling strategy to select additional intra-document image-sentence pairs as positive or negative samples. Furthermore, to recognize the complex pattern in intra-document samples, we propose a Transformer based model to capture fine-grained features and implicitly construct a graph for each document, where concepts in a document are introduced to bridge the representation learning of images and sentences in the context of a document. Experimental results show the effectiveness of our approach to alleviate the bias and learn well-aligned multimodal representations.

cs.CV↗

arXiv2020

Observation of pseudogap in SnSe2 atomic layers grown on graphite

Superconducting metal dichalcogenides (MDCs) present several similarities to the other layered superconductors like cuprates. The superconductivity in atomically thin MDCs has been demonstrated by recent experiments, however, the investigation of the superconductivity intertwined with other orders are scarce. Investigating the pseudogap in atomic layers of MDCs may help to understand the superconducting mechanism for these true two-dimensional (2D) superconducting systems. Herein we report a pseudogap opening in the tunneling spectra of thin layers of SnSe2 epitaxially grown on highly oriented pyrolytic graphite (HOPG) with scanning tunneling microscopy/spectroscopy (STM/STS). A significant V-shaped pseudogap was observed to open near the Fermi level (EF) in the STS. And at elevated temperatures, the gap gradually evolves to a shallow dip. Our experimental observations provide direct evidence of a pseudogap state in the electron-doped SnSe2 atomic layers on the HOPG surface, which may stimulate further exploration of the mechanism of superconductivity at 2D limit in MDCs.

cond-mat.supr-con↗

arXiv2019

Testing the Etherington's distance duality relation at higher redshifts: the combination of radio quasars and gravitational waves

In this paper we analyse the implications of the latest cosmological data sets to test the Etherington's distance duality relation (DDR), which connects the luminosity distance $D_L$ and angular diameter distance $D_A$ at the same redshift. For $D_L$ we consider the simulated data of gravitational waves from the third-generation gravitational wave detector (the Einstein Telescope, ET), which can be considered as standard candles (or standard siren), while the angular diameter distances $D_A$ are derived from the newly-compiled sample of compact radio quasars observed by very-long-baseline interferometry (VLBI), which represents a type of new cosmological standard ruler. Alleviating the absorption and scattering effects of dust in the Universe, this will create a valuable opportunity to directly test DDR at much higher precision with the combination of gravitational wave (GW) and electromagnetic (EM) signals. Our results show that, with the combination of the current radio quasar observations, the duality-distance relation can be verified at the precision of $10^{-2}$. Moreover, the Einstein Telescope ET would produce more robust constraints on the validity of such distance duality relation (at the precision of $10^{-3}$), with a larger sample of compact milliarcsecond radio quasars detected in future VLBI surveys.

astro-ph.CO↗

arXiv2017

AdaDNNs: Adaptive Ensemble of Deep Neural Networks for Scene Text Recognition

Recognizing text in the wild is a really challenging task because of complex backgrounds, various illuminations and diverse distortions, even with deep neural networks (convolutional neural networks and recurrent neural networks). In the end-to-end training procedure for scene text recognition, the outputs of deep neural networks at different iterations are always demonstrated with diversity and complementarity for the target object (text). Here, a simple but effective deep learning method, an adaptive ensemble of deep neural networks (AdaDNNs), is proposed to simply select and adaptively combine classifier components at different iterations from the whole learning system. Furthermore, the ensemble is formulated as a Bayesian framework for classifier weighting and combination. A variety of experiments on several typical acknowledged benchmarks, i.e., ICDAR Robust Reading Competition (Challenge 1, 2 and 4) datasets, verify the surprised improvement from the baseline DNNs, and the effectiveness of AdaDNNs compared with the recent state-of-the-art methods.

cs.CV↗