arXiv subjects

Zheng Liu

Publications and source records attributed to Zheng Liu.

At least 307 records · Page 17Linked to original sources

arXiv2022

Distill-VQ: Learning Retrieval Oriented Vector Quantization By Distilling Knowledge from Dense Embeddings

Vector quantization (VQ) based ANN indexes, such as Inverted File System (IVF) and Product Quantization (PQ), have been widely applied to embedding based document retrieval thanks to the competitive time and memory efficiency. Originally, VQ is learned to minimize the reconstruction loss, i.e., the distortions between the original dense embeddings and the reconstructed embeddings after quantization. Unfortunately, such an objective is inconsistent with the goal of selecting ground-truth documents for the input query, which may cause severe loss of retrieval quality. Recent works identify such a defect, and propose to minimize the retrieval loss through contrastive learning. However, these methods intensively rely on queries with ground-truth documents, whose performance is limited by the insufficiency of labeled data. In this paper, we propose Distill-VQ, which unifies the learning of IVF and PQ within a knowledge distillation framework. In Distill-VQ, the dense embeddings are leveraged as "teachers", which predict the query's relevance to the sampled documents. The VQ modules are treated as the "students", which are learned to reproduce the predicted relevance, such that the reconstructed embeddings may fully preserve the retrieval result of the dense embeddings. By doing so, Distill-VQ is able to derive substantial training signals from the massive unlabeled data, which significantly contributes to the retrieval quality. We perform comprehensive explorations for the optimal conduct of knowledge distillation, which may provide useful insights for the learning of VQ based ANN index. We also experimentally show that the labeled data is no longer a necessity for high-quality vector quantization, which indicates Distill-VQ's strong applicability in practice.

cs.IR↗

arXiv2022

Towards Generalizable Semantic Product Search by Text Similarity Pre-training on Search Click Logs

Recently, semantic search has been successfully applied to e-commerce product search and the learned semantic space(s) for query and product encoding are expected to generalize to unseen queries or products. Yet, whether generalization can conveniently emerge has not been thoroughly studied in the domain thus far. In this paper, we examine several general-domain and domain-specific pre-trained Roberta variants and discover that general-domain fine-tuning does not help generalization, which aligns with the discovery of prior art. Proper domain-specific fine-tuning with clickstream data can lead to better model generalization, based on a bucketed analysis of a publicly available manual annotated query-product pair da

cs.IR↗

arXiv2022

Homointerface planar Josephson junction based on inverse proximity effect

The quality of a superconductor-normal metal-superconductor Josephson junction (JJ) depends crucially on the transparency of the superconductor-normal metal (S/N) interface. We demonstrate a technique for fabricating planar JJs with perfect S/N interfaces. The technique utilizes a strong inverse proximity effect discovered in Al/V$_5$S$_8$ bilayers, by which the Al layer is driven into the resistive state. The highly transparent S/N homointerface and the peculiar normal metal enable the flow of Josephson supercurrent across a 2.9 $μ$m long weak link. Moreover, our JJ exhibits a giant critical current and a large product of the critical current and the normal state resistance.

cond-mat.supr-con↗

arXiv2022

Topological Corner States in Graphene by Bulk and Edge Engineering

Two-dimensional higher-order topology is usually studied in (nearly) particle-hole symmetric models, so that an edge gap can be opened within the bulk one. But more often deviates the edge anticrossing even into the bulk, where corner states are difficult to pinpoint. We address this problem in a graphene-based $\mathbb{Z}_2$ topological insulator with spin-orbit coupling and in-plane magnetization both originating from substrates through a Slater-Koster multi-orbital model. The gapless helical edge modes cross inside the bulk, where is also located the magnetization-induced edge gap. After demonstrating its second-order nontriviality in bulk topology by a series of evidence, we show that a difference in bulk-edge onsite energy can adiabatically tune the position of the crossing/anticrossing of the edge modes to be inside the bulk gap. This can help unambiguously identify two pairs of topological corner states with nonvanishing energy degeneracy for a rhombic flake. We further find that the obtuse-angle pair is more stable than the acute-angle one. These results not only suggest an accessible way to "find" topological corner states, but also provide a higher-order topological version of "bulk-boundary correspondence".

cond-mat.mes-hall↗

arXiv2022

Enhanced versatility of table-top X-rays from van der Waals structures

Van der Waals (vdW) materials have attracted much interest for their myriad unique electronic, mechanical and thermal properties. In particular, they are promising candidates for monochromatic, table-top X-ray sources. This work reveals that the versatility of the table-top vdW X-ray source goes beyond what has been demonstrated so far. By introducing a tilt angle between the vdW structure and the incident electron beam, it is theoretically and experimentally shown that the accessible photon energy range is more than doubled. This allows for greater versatility in real-time tuning of the vdW X-ray source. Furthermore, this work shows that the accessible photon energy range is maximized by simultaneously controlling both the electron energy and the vdW structure tilt. These results should pave the way for highly tunable, compact X-ray sources, with potential applications including hyperspectral X-ray fluoroscopy and X-ray quantum optics.

physics.optics↗

arXiv2022

Progressively Optimized Bi-Granular Document Representation for Scalable Embedding Based Retrieval

Ad-hoc search calls for the selection of appropriate answers from a massive-scale corpus. Nowadays, the embedding-based retrieval (EBR) becomes a promising solution, where deep learning based document representation and ANN search techniques are allied to handle this task. However, a major challenge is that the ANN index can be too large to fit into memory, given the considerable size of answer corpus. In this work, we tackle this problem with Bi-Granular Document Representation, where the lightweight sparse embeddings are indexed and standby in memory for coarse-grained candidate search, and the heavyweight dense embeddings are hosted in disk for fine-grained post verification. For the best of retrieval accuracy, a Progressive Optimization framework is designed. The sparse embeddings are learned ahead for high-quality search of candidates. Conditioned on the candidate distribution induced by the sparse embeddings, the dense embeddings are continuously learned to optimize the discrimination of ground-truth from the shortlisted candidates. Besides, two techniques: the contrastive quantization and the locality-centric sampling are introduced for the learning of sparse and dense embeddings, which substantially contribute to their performances. Thanks to the above features, our method effectively handles massive-scale EBR with strong advantages in accuracy: with up to +4.3% recall gain on million-scale corpus, and up to +17.5% recall gain on billion-scale corpus. Besides, Our method is applied to a major sponsored search platform with substantial gains on revenue (+1.95%), Recall (+1.01%) and CTR (+0.49%). Our code is available at https://github.com/microsoft/BiDR.

cs.IR↗

arXiv2022

A Mutually Reinforced Framework for Pretrained Sentence Embeddings

The lack of labeled data is a major obstacle to learning high-quality sentence embeddings. Recently, self-supervised contrastive learning (SCL) is regarded as a promising way to address this problem. However, the existing works mainly rely on hand-crafted data annotation heuristics to generate positive training samples, which not only call for domain expertise and laborious tuning, but are also prone to the following unfavorable cases: 1) trivial positives, 2) coarse-grained positives, and 3) false positives. As a result, the self-supervision's quality can be severely limited in reality. In this work, we propose a novel framework InfoCSE to address the above problems. Instead of relying on annotation heuristics defined by humans, it leverages the sentence representation model itself and realizes the following iterative self-supervision process: on one hand, the improvement of sentence representation may contribute to the quality of data annotation; on the other hand, more effective data annotation helps to generate high-quality positive samples, which will further improve the current sentence representation model. In other words, the representation learning and data annotation become mutually reinforced, where a strong self-supervision effect can be derived. Extensive experiments are performed based on three benchmark datasets, where notable improvements can be achieved against the existing SCL-based methods.

cs.CL↗

arXiv2022

Direct Light Orbital Angular Momentum Detection in Mid-Infrared based on Type-II Weyl Semimetal TaIrTe4

The capability of direct photocurrent detection of orbital angular momentum (OAM) of light has recently been realized with topological Weyl semimetal, but limited to near infrared wavelength range. The extension of direct OAM detection to midinfrared, a wavelength range that plays important role in a vast range of applications, such as autonomous driving, night vision and motion detection, is challenging and has not yet been realized. This is because most studies of photocurrent responses are not sensitive to the phase information and the photo response is usually very poor in the mid-infrared. In this study, we designed a photodetector based on Type-II Weyl semimetal tantalum iridium tellurides with designed electrode geometries for direct detection of the topological charge of OAM through orbital photogalvanic effect. Our results indicate helical phase gradient of light can be distinguished by a current winding around the optical beam axis with a magnitude proportional to its quantized OAM mode number. The topological enhanced response at mid-infrared of TaIrTe4 further help overcome the low responsivity issues and finally render the direct orbital angular momentum detection capability in mid-infrared. Our study enables on-chip integrated OAM detection, and thus OAM sensitive focal plane arrays in mid-infrared. Such capability triggers new route to explore applications of light carrying OAM, especially that it can crucially promote the performance of many mid-infrared imaging related applications, such as intricate target recognition and night vision.

physics.optics↗

arXiv2022

Uni-Retriever: Towards Learning The Unified Embedding Based Retriever in Bing Sponsored Search

Embedding based retrieval (EBR) is a fundamental building block in many web applications. However, EBR in sponsored search is distinguished from other generic scenarios and technically challenging due to the need of serving multiple retrieval purposes: firstly, it has to retrieve high-relevance ads, which may exactly serve user's search intent; secondly, it needs to retrieve high-CTR ads so as to maximize the overall user clicks. In this paper, we present a novel representation learning framework Uni-Retriever developed for Bing Search, which unifies two different training modes knowledge distillation and contrastive learning to realize both required objectives. On one hand, the capability of making high-relevance retrieval is established by distilling knowledge from the ``relevance teacher model''. On the other hand, the capability of making high-CTR retrieval is optimized by learning to discriminate user's clicked ads from the entire corpus. The two training modes are jointly performed as a multi-objective learning process, such that the ads of high relevance and CTR can be favored by the generated embeddings. Besides the learning strategy, we also elaborate our solution for EBR serving pipeline built upon the substantially optimized DiskANN, where massive-scale EBR can be performed with competitive time and memory efficiency, and accomplished in high-quality. We make comprehensive offline and online experiments to evaluate the proposed techniques, whose findings may provide useful insights for the future development of EBR systems. Uni-Retriever has been mainstreamed as the major retrieval path in Bing's production thanks to the notable improvements on the representation and EBR serving quality.

cs.IR↗

arXiv2022

1T-FeS$_2$$:$ a new type of two-dimensional metallic ferromagnet

Discovery of intrinsic two-dimensional (2D) magnetic materials is crucial for understanding the fundamentals of 2D magnetism and realizing next-generation magnetoelectronic and magneto-optical devices. Although significant efforts have been devoted to identifying 2D magnetism by exfoliating bulk magnetic layered materials, seldom studies are performed to synthesize ultra-thin magnetic materials directly for non-layered magnetic materials. Here, we report the successful synthesis of a new type of theoretically proposed 2D metallic ferromagnet 1T FeS2, through the molten-salt-assisted chemical vapor deposition (CVD) method. The long-range 2D ferromagnetic order is confirmed by the observation of a large anomalous Hall effect (AHE) and a hysteretic magnetoresistance. The experimentally detected out-of-plane ferromagnetic ordering is theoretically suported with Stoner criterion. Our findings open up new possibilities to search novel 2D ferromagnets in non-layered compounds and render opportunities for realizing realistic ultra-thin spintronic devices.

cond-mat.mtrl-sci↗

arXiv2022

Second-Order Topological Insulator in van der Waals Heterostructures of CoBr$_2$/Pt$_2$HgSe$_3$/CoBr$_2$

Second-order topological insulator, which has (d-2)-dimensional topological hinge or corner states, has been observed in three-dimensional materials, but has yet not been observed in two-dimensional system. In this Letter, we theoretically propose the realization of second-order topological insulator in the van der Waals heterostructure of CoBr$_2$/Pt$_2$HgSe$_3$/CoBr$_2$. Pt$_2$HgSe$_3$ is a large gap $\mathbb{Z}_2$ topological insulator. With in-plane exchange field from neighboring CoBr$_2$, a large band gap above 70 meV opens up at the edge. The corner states, which are robust against edge disorders and irregular shapes, are confirmed in the nanoflake. We further show that the second-order topological states can also be realized in the heterostructure of jacutingaite family $\mathbb{Z}_2$ topological insulators. We believe that our work will be beneficial for the experimental realization of second-order topological insulators in van der Waals layered materials.

cond-mat.mes-hall↗

arXiv2022

Effect of Stacking Order on the Electronic State of 1T-TaS$_2$

New theoretical proposals and experimental findings on transition metal dichalcogenide 1T-TaS$_2$ have revived interests in its possible Mott insulating state. We perform a comprehensive scanning tunneling microscopy and spectroscopy experiment on different single-step areas in pristine 1T-TaS$_2$. After accurately determining the relative displacement of Star-of-David super-lattices in two layers, we find different stacking orders can correspond to the similar large-gap spectrum on the upper terrace. When the measurement is performed away from the step edge, the large gap spectrum can always be maintained. The stacking order seems rarely disturb the large-gap spectrum in the ideal bulk material. We conclude that the large insulating gap is from the single-layer property, which is a correlation-induced Mott gap based on the single-band Hubbard model. Specific stacking orders can perturb the state and induce a small-gap or metallic spectrum for a limited area around the step edge, which we attribute to a surface and edge phenomenon. Our work provides more evidence about the surface electronic state and deepens our understanding of the Mott insulating state in 1T-TaS$_2$.

cond-mat.str-el↗

arXiv2022

Reconcile the Bulk Metallic and Surface Insulating state in 1T-TaSe$_2$

The transition metal dichalcogenides 1T-TaS$_2$ and 1T-TaSe$_2$ have been extensively studied for the complicated correlated electronic properties. The origin of different surface electronic states remains controversial. We apply scanning tunneling microscopy and spectroscopy to restudy the surface electronic state of bulk 1T-TaSe$_2$. Both insulating and metallic states are identified in different areas of the same sample. The insulating state is similar to that in 1T-TaS$_2$, concerning both the dI/dV spectrum and the orbital texture. With further investigations in single-step areas, the discrepancy of electronic states is found to be associated with different stacking orders. The insulating state is most possibly a single-layer property, modulated to a metallic state in some particular stacking orders. Both the metallic and large-gap insulating spectra, together with their corresponding stacking orders, are dominant in 1T-TaSe$_2$. The connected metallic areas lead to the metallic transport behavior. We then reconcile the bulk metallic and surface insulating state in 1T-TaSe$_2$. The rich phenomena in 1T-TaSe$_2$ deepen our understanding of the correlated electronic state in bulk 1T-TaSe$_2$ and 1T-TaS$_2$.

cond-mat.str-el↗

arXiv2021

An Adaptive Graph Pre-training Framework for Localized Collaborative Filtering

Graph neural networks (GNNs) have been widely applied in the recommendation tasks and have obtained very appealing performance. However, most GNN-based recommendation methods suffer from the problem of data sparsity in practice. Meanwhile, pre-training techniques have achieved great success in mitigating data sparsity in various domains such as natural language processing (NLP) and computer vision (CV). Thus, graph pre-training has the great potential to alleviate data sparsity in GNN-based recommendations. However, pre-training GNNs for recommendations face unique challenges. For example, user-item interaction graphs in different recommendation tasks have distinct sets of users and items, and they often present different properties. Therefore, the successful mechanisms commonly used in NLP and CV to transfer knowledge from pre-training tasks to downstream tasks such as sharing learned embeddings or feature extractors are not directly applicable to existing GNN-based recommendations models. To tackle these challenges, we delicately design an adaptive graph pre-training framework for localized collaborative filtering (ADAPT). It does not require transferring user/item embeddings, and is able to capture both the common knowledge across different graphs and the uniqueness for each graph. Extensive experimental results have demonstrated the effectiveness and superiority of ADAPT.

cs.IR↗

arXiv2021

Semi-Supervised Variational User Identity Linkage via Noise-Aware Self-Learning

User identity linkage, which aims to link identities of a natural person across different social platforms, has attracted increasing research interest recently. Existing approaches usually first embed the identities as deterministic vectors in a shared latent space, and then learn a classifier based on the available annotations. However, the formation and characteristics of real-world social platforms are full of uncertainties, which makes these deterministic embedding based methods sub-optimal. In addition, it is intractable to collect sufficient linkage annotations due to the tremendous gaps between different platforms. Semi-supervised models utilize the unlabeled data to help capture the intrinsic data distribution, which are more promising in practical usage. However, the existing semi-supervised linkage methods heavily rely on the heuristically defined similarity measurements to incorporate the innate closeness between labeled and unlabeled samples. Such manually designed assumptions may not be consistent with the actual linkage signals and further introduce the noises. To address the mentioned limitations, in this paper we propose a novel Noise-aware Semi-supervised Variational User Identity Linkage (NSVUIL) model. Specifically, we first propose a novel supervised linkage module to incorporate the available annotations. Each social identity is represented by a Gaussian distribution in the Wasserstein space to simultaneously preserve the fine-grained social profiles and model the uncertainty of identities. Then, a noise-aware self-learning module is designed to faithfully augment the few available annotations, which is capable of filtering noises from the pseudo-labels generated by the supervised module.

cs.SI↗

arXiv2021

Pre-training Recommender Systems via Reinforced Attentive Multi-relational Graph Neural Network

Recently, Graph Neural Networks (GNNs) have proven their effectiveness for recommender systems. Existing studies have applied GNNs to capture collaborative relations in the data. However, in real-world scenarios, the relations in a recommendation graph can be of various kinds. For example, two movies may be associated either by the same genre or by the same director/actor. If we use a single graph to elaborate all these relations, the graph can be too complex to process. To address this issue, we bring the idea of pre-training to process the complex graph step by step. Based on the idea of divide-and-conquer, we separate the large graph into three sub-graphs: user graph, item graph, and user-item interaction graph. Then the user and item embeddings are pre-trained from user and item graphs, respectively. To conduct pre-training, we construct the multi-relational user graph and item graph, respectively, based on their attributes. In this paper, we propose a novel Reinforced Attentive Multi-relational Graph Neural Network (RAM-GNN) to the pre-train user and item embeddings on the user and item graph prior to the recommendation step. Specifically, we design a relation-level attention layer to learn the importance of different relations. Next, a Reinforced Neighbor Sampler (RNS) is applied to search the optimal filtering threshold for sampling top-k similar neighbors in the graph, which avoids the over-smoothing issue. We initialize the recommendation model with the pre-trained user/item embeddings. Finally, an aggregation-based GNN model is utilized to learn from the collaborative relations in the user-item interaction graph and provide recommendations. Our experiments demonstrate that RAM-GNN outperforms other state-of-the-art graph-based recommendation models and multi-relational graph neural networks.

cs.IR↗

arXiv2021

Certifiable Artificial Intelligence Through Data Fusion

This paper reviews and proposes concerns in adopting, fielding, and maintaining artificial intelligence (AI) systems. While the AI community has made rapid progress, there are challenges in certifying AI systems. Using procedures from design and operational test and evaluation, there are opportunities towards determining performance bounds to manage expectations of intended use. A notional use case is presented with image data fusion to support AI object recognition certifiability considering precision versus distance.

cs.AI↗

arXiv2021

An overview of sensing platform-technological aspects for vector magnetic measurement: a case study of the application in different scenarios

Magnetic sensing platform techniques have been used in many years in an attempt to better evaluate the likelihood of recoverable hydrocarbon reservoirs by determining the depth and pattern of sedimentary rock formations containing magnetic minerals, such as magnetite. Utilizing airplanes, large-area magnetic surveys have been conducted to estimate, for example, the depth of igneous rock and the thickness of sedimentary rock formations. In this case, the vector magnetic survey method can simultaneously obtain the modulus and direction information of the Earth's magnetic field, which can effectively reduce the multiplicity on data inversion, contribute to the quantitative interpretation of the magnetic body and obtain more precise information and characteristics of magnetic field resource, so as to improve the detection resolution and positioning accuracy of the underground target body. This paper presents a state-of-the-art review of the application situations, the technical features, and the development of the vector magnetic sensing platform-technical aspects for different application scenarios, i.e., ground, wells, marine, airborne, and satellites, respectively. The potential of multi-survey sensing platform technique fusion for magnetic field detection is also discussed.

physics.geo-ph↗