arXiv subjects

Zheng Liu

Publications and source records attributed to Zheng Liu.

At least 289 records · Page 16Linked to original sources

arXiv2022

CDSM: Cascaded Deep Semantic Matching on Textual Graphs Leveraging Ad-hoc Neighbor Selection

Deep semantic matching aims to discriminate the relationship between documents based on deep neural networks. In recent years, it becomes increasingly popular to organize documents with a graph structure, then leverage both the intrinsic document features and the extrinsic neighbor features to derive discrimination. Most of the existing works mainly care about how to utilize the presented neighbors, whereas limited effort is made to filter appropriate neighbors. We argue that the neighbor features could be highly noisy and partially useful. Thus, a lack of effective neighbor selection will not only incur a great deal of unnecessary computation cost, but also restrict the matching accuracy severely. In this work, we propose a novel framework, Cascaded Deep Semantic Matching (CDSM), for accurate and efficient semantic matching on textual graphs. CDSM is highlighted for its two-stage workflow. In the first stage, a lightweight CNN-based ad-hod neighbor selector is deployed to filter useful neighbors for the matching task with a small computation cost. We design both one-step and multi-step selection methods. In the second stage, a high-capacity graph-based matching network is employed to compute fine-grained relevance scores based on the well-selected neighbors. It is worth noting that CDSM is a generic framework which accommodates most of the mainstream graph-based semantic matching networks. The major challenge is how the selector can learn to discriminate the neighbors usefulness which has no explicit labels. To cope with this problem, we design a weak-supervision strategy for optimization, where we train the graph-based matching network at first and then the ad-hoc neighbor selector is learned on top of the annotations from the matching network.

cs.IR↗

arXiv2022

Valley-polarized quantum anomalous Hall effect in van der Waals heterostructures based on monolayer jacutingaite family materials

We numerically study the general valley polarization and anomalous Hall effect in van der Waals (vdW) heterostructures based on monolayer jacutingaite family materials Pt$_{2}$AX$_{3}$ (A = Hg, Cd, Zn; X = S, Se, Te). We perform a systematic study on the atomic, electronic, and topological properties of vdW heterostructures composed of monolayer Pt$_{2}$AX$_{3}$ and two-dimensional ferromagnetic insulators. We show that four kinds of vdW heterostructures exhibit valley-polarized quantum anomalous Hall phase, i.e., Pt$_{2}$HgS$_{3}$/NiBr$_{2}$, Pt$_{2}$HgSe$_{3}$/CoBr$_{2}$, Pt$_{2}$HgSe$_{3}$/NiBr$_{2}$, and Pt$_{2}$ZnS$_{3}$/CoBr$_{2}$, with a maximum valley splitting of 134.2 meV in Pt$_{2}$HgSe$_{3}$/NiBr$_{2}$ and sizable global band gap of 58.8 meV in Pt$_{2}$HgS$_{3}$/NiBr$_{2}$. Our findings demonstrate an ideal platform to implement applications on topological valleytronics.

cond-mat.mes-hall↗

arXiv2022

RetroMAE v2: Duplex Masked Auto-Encoder For Pre-Training Retrieval-Oriented Language Models

To better support retrieval applications such as web search and question answering, growing effort is made to develop retrieval-oriented language models. Most of the existing works focus on improving the semantic representation capability for the contextualized embedding of [CLS] token. However, recent study shows that the ordinary tokens besides [CLS] may provide extra information, which helps to produce a better representation effect. As such, it's necessary to extend the current methods where all contextualized embeddings can be jointly pre-trained for the retrieval tasks. With this motivation, we propose a new pre-training method: duplex masked auto-encoder, a.k.a. DupMAE, which targets on improving the semantic representation capacity for the contextualized embeddings of both [CLS] and ordinary tokens. It introduces two decoding tasks: one is to reconstruct the original input sentence based on the [CLS] embedding, the other one is to minimize the bag-of-words loss (BoW) about the input sentence based on the entire ordinary tokens' embeddings. The two decoding losses are added up to train a unified encoding model. The embeddings from [CLS] and ordinary tokens, after dimension reduction and aggregation, are concatenated as one unified semantic representation for the input. DupMAE is simple but empirically competitive: with a small decoding cost, it substantially contributes to the model's representation capability and transferability, where remarkable improvements are achieved on MS MARCO and BEIR benchmarks.

cs.CL↗

arXiv2022

Mitigating Frequency Bias in Next-Basket Recommendation via Deconfounders

Recent studies on Next-basket Recommendation (NBR) have achieved much progress by leveraging Personalized Item Frequency (PIF) as one of the main features, which measures the frequency of the user's interactions with the item. However, taking the PIF as an explicit feature incurs bias towards frequent items. Items that a user purchases frequently are assigned higher weights in the PIF-based recommender system and appear more frequently in the personalized recommendation list. As a result, the system will lose the fairness and balance between items that the user frequently purchases and items that the user never purchases. We refer to this systematic bias on personalized recommendation lists as frequency bias, which narrows users' browsing scope and reduces the system utility. We adopt causal inference theory to address this issue. Considering the influence of historical purchases on users' future interests, the user and item representations can be viewed as unobserved confounders in the causal diagram. In this paper, we propose a deconfounder model named FENDER (Frequency-aware Deconfounder for Next-basket Recommendation) to mitigate the frequency bias. With the deconfounder theory and the causal diagram we propose, FENDER decomposes PIF with a neural tensor layer to obtain substitute confounders for users and items. Then, FENDER performs unbiased recommendations considering the effect of these substitute confounders. Experimental results demonstrate that FENDER has derived diverse and fair results compared to ten baseline models on three datasets while achieving competitive performance. Further experiments illustrate how FENDER balances users' historical purchases and potential interests.

cs.IR↗

arXiv2022

Collaborative Bi-Aggregation for Directed Graph Embedding

Directed graphs model asymmetric relationships between nodes and research on directed graph embedding is of great significance in downstream graph analysis and inference. Learning source and target embedding of nodes separately to preserve edge asymmetry has become the dominant approach, but also poses challenge for learning representations of low or even zero in/out degree nodes that are ubiquitous in sparse graphs. In this paper, a collaborative bi-directional aggregation method (COBA) for directed graphs embedding is proposed by introducing spatial-based graph convolution. Firstly, the source and target embeddings of the central node are learned by aggregating from the counterparts of the source and target neighbors, respectively; Secondly, the source/target embeddings of the zero in/out degree central nodes are enhanced by aggregating the counterparts of opposite-directional neighbors (i.e. target/source neighbors); Finally, source and target embeddings of the same node are correlated to achieve collaborative aggregation. Extensive experiments on real-world datasets demonstrate that the COBA comprehensively outperforms state-of-the-art methods on multiple tasks and meanwhile validates the effectiveness of proposed aggregation strategies.

cs.SI↗

arXiv2022

Mitigating Health Disparities in EHR via Deconfounder

Health disparities, or inequalities between different patient demographics, are becoming crucial in medical decision-making, especially in Electronic Health Record (EHR) predictive modeling. To ensure the fairness of sensitive attributes, conventional studies mainly adopt calibration or re-weighting methods to balance the performance on among different demographic groups. However, we argue that these methods have some limitations. First, these methods usually mean a trade-off between the model's performance and fairness. Second, many methods completely attribute unfairness to the data collection process, which lacks substantial evidence. In this paper, we provide an empirical study to discover the possibility of using deconfounder to address the disparity issue in healthcare. Our study can be summarized in two parts. The first part is a pilot study demonstrating the exacerbation of disparity when unobserved confounders exist. The second part proposed a novel framework, Parity Medical Deconfounder (PriMeD), to deal with the disparity issue in healthcare datasets. Inspired by the deconfounder theory, PriMeD adopts a Conditional Variational Autoencoder (CVAE) to learn latent factors (substitute confounders) for observational data, and extensive experiments are provided to show its effectiveness.

cs.LG↗

arXiv2022

RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encoder

Despite pre-training's progress in many important NLP tasks, it remains to explore effective pre-training strategies for dense retrieval. In this paper, we propose RetroMAE, a new retrieval oriented pre-training paradigm based on Masked Auto-Encoder (MAE). RetroMAE is highlighted by three critical designs. 1) A novel MAE workflow, where the input sentence is polluted for encoder and decoder with different masks. The sentence embedding is generated from the encoder's masked input; then, the original sentence is recovered based on the sentence embedding and the decoder's masked input via masked language modeling. 2) Asymmetric model structure, with a full-scale BERT like transformer as encoder, and a one-layer transformer as decoder. 3) Asymmetric masking ratios, with a moderate ratio for encoder: 15~30%, and an aggressive ratio for decoder: 50~70%. Our framework is simple to realize and empirically competitive: the pre-trained models dramatically improve the SOTA performances on a wide range of dense retrieval benchmarks, like BEIR and MS MARCO. The source code and pre-trained models are made publicly available at https://github.com/staoxiao/RetroMAE so as to inspire more interesting research.

cs.CL↗

arXiv2022

Large-Scale LiDAR Consistent Mapping using Hierachical LiDAR Bundle Adjustment

Reconstructing an accurate and consistent large-scale LiDAR point cloud map is crucial for robotics applications. The existing solution, pose graph optimization, though it is time-efficient, does not directly optimize the mapping consistency. LiDAR bundle adjustment (BA) has been recently proposed to resolve this issue; however, it is too time-consuming on large-scale maps. To mitigate this problem, this paper presents a globally consistent and efficient mapping method suitable for large-scale maps. Our proposed work consists of a bottom-up hierarchical BA and a top-down pose graph optimization, which combines the advantages of both methods. With the hierarchical design, we solve multiple BA problems with a much smaller Hessian matrix size than the original BA; with the pose graph optimization, we smoothly and efficiently update the LiDAR poses. The effectiveness and robustness of our proposed approach have been validated on multiple spatially and timely large-scale public spinning LiDAR datasets, i.e., KITTI, MulRan and Newer College, and self-collected solid-state LiDAR datasets under structured and unstructured scenes. With proper setups, we demonstrate our work could generate a globally consistent map with around 12% of the sequence time.

cs.RO↗

arXiv2022

Pre-training for Information Retrieval: Are Hyperlinks Fully Explored?

Recent years have witnessed great progress on applying pre-trained language models, e.g., BERT, to information retrieval (IR) tasks. Hyperlinks, which are commonly used in Web pages, have been leveraged for designing pre-training objectives. For example, anchor texts of the hyperlinks have been used for simulating queries, thus constructing tremendous query-document pairs for pre-training. However, as a bridge across two web pages, the potential of hyperlinks has not been fully explored. In this work, we focus on modeling the relationship between two documents that are connected by hyperlinks and designing a new pre-training objective for ad-hoc retrieval. Specifically, we categorize the relationships between documents into four groups: no link, unidirectional link, symmetric link, and the most relevant symmetric link. By comparing two documents sampled from adjacent groups, the model can gradually improve its capability of capturing matching signals. We propose a progressive hyperlink predication ({PHP}) framework to explore the utilization of hyperlinks in pre-training. Experimental results on two large-scale ad-hoc retrieval datasets and six question-answering datasets demonstrate its superiority over existing pre-training methods.

cs.IR↗

arXiv2022

Fighting Sybils in Airdrops

Airdrop is a crucial concept in tokenomics. Startups of decentralized applications (DApps) reward early supporters by airdropping newly issued tokens up to a certain amount as a free giveaway. This naturally induces greedy hackers, called Sybils, to create multiple accounts for more shares. Most airdrops have prerequisites for qualification, in which utilizing these DApps is unsurprisingly the principal. One particular characteristic of DApps is to implement users' interactions with them in the form of token transfer transactions or smart contract calling transactions on public blockchains. We argue that these individual transactions could reveal underlying signatures of their sending accounts. Specifically, accounts controlled by the same Sybil may exhibit some common behaviors. A careful analysis of Sybil's behaviors shows that accounts controlled by the same Sybil may produce similar DApp activities and regular token transfer patterns. We model the transactions as graphs by representing accounts as vertices and transactions as edges. When multiple accounts receive tokens from the same Sybil to conduct interactions with DApps, we inspect the graphs for these activities and patterns to detect suspicious accounts. We demonstrate the effectiveness of the proposed method in a recent airdrop by presenting the suspicious accounts controlled by Sybils. All the detected accounts exhibit similar interaction activities and regular transfer patterns.

cs.CR↗

arXiv2022

Coarse-to-Fine: Hierarchical Multi-task Learning for Natural Language Understanding

Generalized text representations are the foundation of many natural language understanding tasks. To fully utilize the different corpus, it is inevitable that models need to understand the relevance among them. However, many methods ignore the relevance and adopt a single-channel model (a coarse paradigm) directly for all tasks, which lacks enough rationality and interpretation. In addition, some existing works learn downstream tasks by stitches skill block(a fine paradigm), which might cause irrationalresults due to its redundancy and noise. Inthis work, we first analyze the task correlation through three different perspectives, i.e., data property, manual design, and model-based relevance, based on which the similar tasks are grouped together. Then, we propose a hierarchical framework with a coarse-to-fine paradigm, with the bottom level shared to all the tasks, the mid-level divided to different groups, and the top-level assigned to each of the tasks. This allows our model to learn basic language properties from all tasks, boost performance on relevant tasks, and reduce the negative impact from irrelevant tasks. Our experiments on 13 benchmark datasets across five natural language understanding tasks demonstrate the superiority of our method.

cs.CL↗

arXiv2022

Error-Aware Spatial Ensembles for Video Frame Interpolation

Video frame interpolation~(VFI) algorithms have improved considerably in recent years due to unprecedented progress in both data-driven algorithms and their implementations. Recent research has introduced advanced motion estimation or novel warping methods as the means to address challenging VFI scenarios. However, none of the published VFI works considers the spatially non-uniform characteristics of the interpolation error (IE). This work introduces such a solution. By closely examining the correlation between optical flow and IE, the paper proposes novel error prediction metrics that partition the middle frame into distinct regions corresponding to different IE levels. Building upon this IE-driven segmentation, and through the use of novel error-controlled loss functions, it introduces an ensemble of spatially adaptive interpolation units that progressively processes and integrates the segmented regions. This spatial ensemble results in an effective and computationally attractive VFI solution. Extensive experimentation on popular video interpolation benchmarks indicates that the proposed solution outperforms the current state-of-the-art (SOTA) in applications of current interest.

cs.CV↗

arXiv2022

Layer Zero-Line Modes in Antiferromagnetic Topological Insulators

Recently, the magnetic domain walls have been experimentally observed in antiferromagnetic topological insulators MnBi$_2$Te$_4$, where we find that the topological zero-line modes (ZLMs) appear along the domain walls. Here, we theoretically demonstrate that these ZLMs are layer-dependent in MnBi$_2$Te$_4$ multilayers. For domain walls with out-of-plane ferromagnetism, we find that ZLMs are equally distributed in the odd-number layers. When domain walls possess in-plane magnetization, the ZLMs can also exist in even-number layers due to in-plane mirror-symmetry breaking. Moreover, the conductive channels are mainly distributed in the outermost layers with increasing layer thickness. Our findings lay out a strategy in manipulating ZLMs and also can be utilized to distinguish the corresponding magnetic structures.

cond-mat.mes-hall↗

arXiv2022

Extensibility of Hohenberg-Kohn Theorem to general quantum systems

Hohenberg-Kohn (HK) theorem is a cornerstone of modern electronic structure calculations. For interacting electrons, given that the internal part of the Hamiltonian ($\hat H_{int}$), containing the kinetic energy and Couloumb interaction of electrons, has a fixed form, the theorem states that when the electrons are subject to an external electrostatic field, the ground-state density can inversely determine the field, and thus the full Hamiltonian completely. For a general quantum system, a HK-type Hamiltonian in the form of $\hat H_{hk}\{g_i\}=\hat H_{int}+\sum_i g_i \hat O_i$ can always be defined, by grouping those terms with fixed or preknown coefficients into $\hat H_{int}$, and factorizing the remaining as superposition of a set of Hermitian operators $\{\hat O_i\}$. We ask whether the HK theorem can be extended, so that the ground-state expectation values of $\{\hat O_i\}$ as the generalized density can in principle be used as the fundamental variables determining all the properties of the system. We show that the question can be addressed by introducing the concept of generalized density correlation matrix (GDCM) defined with respect to the $\{\hat O_i\}$ operators. The invertibility of the GDCM represents a mathematically rigorous and practically useful criterion for the extension of HK theorem to be valid. We apply this criterion to several representative systems, including the quantum Ising dimer, the frustration-free systems, N-level quantum systems with fixed inter-level transition amplitude and tunable level energies, and a fermionic Hubbard chain with inhomogeneous on-site interactions. We suggest that for a finite-size system, finding an invertible GDCM under one single $\{g_i\}$ configuration is typically sufficient to establish the generic extensibility of the HK theorem in the entire parameter space.

quant-ph↗

arXiv2022

Asymmetric Fraunhofer pattern in Josephson junctions from heterodimensional superlattice V$_5$S$_8$

Introduction of spin-orbit coupling (SOC) in a Josephson junction (JJ) gives rise to unusual Josephson effects. We investigate JJs based on a newly discovered heterodimensional superlattice V$_5$S$_8$ with a special form of SOC. The unique homointerface of our JJs enables elimination of extrinsic effects due to interfaces and disorder. We observe asymmetric Fraunhofer patterns with respect to both the perpendicular magnetic field and the current. The asymmetry is influenced by an in-plane magnetic field. Analysis of the pattern points to a nontrivial spatial distribution of the Josephson current that is intrinsic to the SOC in V$_5$S$_8$.

cond-mat.supr-con↗

arXiv2022

Ada-Ranker: A Data Distribution Adaptive Ranking Paradigm for Sequential Recommendation

A large-scale recommender system usually consists of recall and ranking modules. The goal of ranking modules (aka rankers) is to elaborately discriminate users' preference on item candidates proposed by recall modules. With the success of deep learning techniques in various domains, we have witnessed the mainstream rankers evolve from traditional models to deep neural models. However, the way that we design and use rankers remains unchanged: offline training the model, freezing the parameters, and deploying it for online serving. Actually, the candidate items are determined by specific user requests, in which underlying distributions (e.g., the proportion of items for different categories, the proportion of popular or new items) are highly different from one another in a production environment. The classical parameter-frozen inference manner cannot adapt to dynamic serving circumstances, making rankers' performance compromised. In this paper, we propose a new training and inference paradigm, termed as Ada-Ranker, to address the challenges of dynamic online serving. Instead of using parameter-frozen models for universal serving, Ada-Ranker can adaptively modulate parameters of a ranker according to the data distribution of the current group of item candidates. We first extract distribution patterns from the item candidates. Then, we modulate the ranker by the patterns to make the ranker adapt to the current data distribution. Finally, we use the revised ranker to score the candidate list. In this way, we empower the ranker with the capacity of adapting from a global model to a local model which better handles the current task.

cs.IR↗

arXiv2022

A Novel Underwater Image Enhancement and Improved Underwater Biological Detection Pipeline

For aquaculture resource evaluation and ecological environment monitoring, automatic detection and identification of marine organisms is critical. However, due to the low quality of underwater images and the characteristics of underwater biological, a lack of abundant features may impede traditional hand-designed feature extraction approaches or CNN-based object detection algorithms, particularly in complex underwater environment. Therefore, the goal of this paper is to perform object detection in the underwater environment. This paper proposed a novel method for capturing feature information, which adds the convolutional block attention module (CBAM) to the YOLOv5 backbone. The interference of underwater creature characteristics on object characteristics is decreased, and the output of the backbone network to object information is enhanced. In addition, the self-adaptive global histogram stretching algorithm (SAGHS) is designed to eliminate the degradation problems such as low contrast and color loss caused by underwater environmental information to better restore image quality. Extensive experiments and comprehensive evaluation on the URPC2021 benchmark dataset demonstrate the effectiveness and adaptivity of our methods. Beyond that, this paper conducts an exhaustive analysis of the role of training data on performance.

cs.CV↗

arXiv2022

Strong Neel ordering and luminescence correlation in a two-dimensional antiferromagnet

Magneto-optical effect has been widely used in light modulation, optical sensing and information storage. Recently discovered two-dimensional (2D) van der Waals layered magnets are considered as promising platforms for investigating novel magneto-optical phenomena and devices, due to the long-range magnetic ordering down to atomically-thin thickness, rich species and tunable properties. However, majority 2D antiferromagnets suffer from low luminescence efficiency which hinders their magneto-optical investigations and applications. Here, we uncover strong light-magnetic ordering interactions in 2D antiferromagnetic MnPS3 utilizing a newly-emerged near-infrared photoluminescence (PL) mode far below its intrinsic bandgap. This ingap PL mode shows strong correlation with the Neel ordering and persists down to monolayer thickness. Combining the DFT, STEM and XPS, we illustrate the origin of the PL mode and its correlation with Neel ordering, which can be attributed to the oxygen ion-mediated states. Moreover, the PL strength can be further tuned and enhanced using ultraviolet-ozone treatment. Our studies offer an effective approach to investigate light-magnetic ordering interactions in 2D antiferromagnetic semiconductors.

physics.optics↗