arXiv subjects

Zheng Liu

Publications and source records attributed to Zheng Liu.

At least 271 records · Page 15Linked to original sources

arXiv2023

PCDNF: Revisiting Learning-based Point Cloud Denoising via Joint Normal Filtering

Recovering high quality surfaces from noisy point clouds, known as point cloud denoising, is a fundamental yet challenging problem in geometry processing. Most of the existing methods either directly denoise the noisy input or filter raw normals followed by updating point positions. Motivated by the essential interplay between point cloud denoising and normal filtering, we revisit point cloud denoising from a multitask perspective, and propose an end-to-end network, named PCDNF, to denoise point clouds via joint normal filtering. In particular, we introduce an auxiliary normal filtering task to help the overall network remove noise more effectively while preserving geometric features more accurately. In addition to the overall architecture, our network has two novel modules. On one hand, to improve noise removal performance, we design a shape-aware selector to construct the latent tangent space representation of the specific point by comprehensively considering the learned point and normal features and geometry priors. On the other hand, point features are more suitable for describing geometric details, and normal features are more conducive for representing geometric structures (e.g., sharp edges and corners). Combining point and normal features allows us to overcome their weaknesses. Thus, we design a feature refinement module to fuse point and normal features for better recovering geometric information. Extensive evaluations, comparisons, and ablation studies demonstrate that the proposed method outperforms state-of-the-arts for both point cloud denoising and normal filtering.

cs.CV↗

arXiv2023

RetroMAE-2: Duplex Masked Auto-Encoder For Pre-Training Retrieval-Oriented Language Models

To better support information retrieval tasks such as web search and open-domain question answering, growing effort is made to develop retrieval-oriented language models, e.g., RetroMAE and many others. Most of the existing works focus on improving the semantic representation capability for the contextualized embedding of the [CLS] token. However, recent study shows that the ordinary tokens besides [CLS] may provide extra information, which help to produce a better representation effect. As such, it's necessary to extend the current methods where all contextualized embeddings can be jointly pre-trained for the retrieval tasks. In this work, we propose a novel pre-training method called Duplex Masked Auto-Encoder, a.k.a. DupMAE. It is designed to improve the quality of semantic representation where all contextualized embeddings of the pre-trained model can be leveraged. It takes advantage of two complementary auto-encoding tasks: one reconstructs the input sentence on top of the [CLS] embedding; the other one predicts the bag-of-words feature of the input sentence based on the ordinary tokens' embeddings. The two tasks are jointly conducted to train a unified encoder, where the whole contextualized embeddings are aggregated in a compact way to produce the final semantic representation. DupMAE is simple but empirically competitive: it substantially improves the pre-trained model's representation capability and transferability, where superior retrieval performances can be achieved on popular benchmarks, like MS MARCO and BEIR.

cs.CL↗

arXiv2023

An Efficient Plane Extraction Approach for Bundle Adjustment on LiDAR Point clouds

Bundle adjustment (BA) on LiDAR point clouds has been extensively investigated in recent years due to its ability to optimize multiple poses together, resulting in high accuracy and global consistency for point cloud. However, the accuracy and speed of LiDAR bundle adjustment depend on the quality of plane extraction, which provides point association for LiDAR BA. In this study, we propose a novel and efficient voxel-based approach for plane extraction that is specially designed to provide point association for LiDAR bundle adjustment. To begin, we partition the space into multiple voxels of a fixed size and then split these root voxels based on whether the points are on the same plane, using an octree structure. We also design a novel plane determination method based on principle component analysis (PCA), which segments the points into four even quarters and compare their minimum eigenvalues with that of the initial point cloud. Finally, we adopt a plane merging method to prevent too many small planes from being in a single voxel, which can increase the optimization time required for BA. Our experimental results on HILTI demonstrate that our approach achieves the best precision and least time cost compared to other plane extraction methods.

cs.CV↗

arXiv2023

Constructing Tree-based Index for Efficient and Effective Dense Retrieval

Recent studies have shown that Dense Retrieval (DR) techniques can significantly improve the performance of first-stage retrieval in IR systems. Despite its empirical effectiveness, the application of DR is still limited. In contrast to statistic retrieval models that rely on highly efficient inverted index solutions, DR models build dense embeddings that are difficult to be pre-processed with most existing search indexing systems. To avoid the expensive cost of brute-force search, the Approximate Nearest Neighbor (ANN) algorithm and corresponding indexes are widely applied to speed up the inference process of DR models. Unfortunately, while ANN can improve the efficiency of DR models, it usually comes with a significant price on retrieval performance. To solve this issue, we propose JTR, which stands for Joint optimization of TRee-based index and query encoding. Specifically, we design a new unified contrastive learning loss to train tree-based index and query encoder in an end-to-end manner. The tree-based negative sampling strategy is applied to make the tree have the maximum heap property, which supports the effectiveness of beam search well. Moreover, we treat the cluster assignment as an optimization problem to update the tree-based index that allows overlapped clustering. We evaluate JTR on numerous popular retrieval benchmarks. Experimental results show that JTR achieves better retrieval performance while retaining high system efficiency compared with widely-adopted baselines. It provides a potential solution to balance efficiency and effectiveness in neural retrieval system designs.

cs.IR↗

arXiv2023

WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpus

In this paper, we introduce a new NLP task -- generating short factual articles with references for queries by mining supporting evidence from the Web. In this task, called WebBrain, the ultimate goal is to generate a fluent, informative, and factually-correct short article (e.g., a Wikipedia article) for a factual query unseen in Wikipedia. To enable experiments on WebBrain, we construct a large-scale dataset WebBrain-Raw by extracting English Wikipedia articles and their crawlable Wikipedia references. WebBrain-Raw is ten times larger than the previous biggest peer dataset, which can greatly benefit the research community. From WebBrain-Raw, we construct two task-specific datasets: WebBrain-R and WebBrain-G, which are used to train in-domain retriever and generator, respectively. Besides, we empirically analyze the performances of the current state-of-the-art NLP techniques on WebBrain and introduce a new framework ReGen, which enhances the generation factualness by improved evidence retrieval and task-specific pre-training for generation. Experiment results show that ReGen outperforms all baselines in both automatic and human evaluations.

cs.CL↗

arXiv2023

Cooperative Retriever and Ranker in Deep Recommenders

Deep recommender systems (DRS) are intensively applied in modern web services. To deal with the massive web contents, DRS employs a two-stage workflow: retrieval and ranking, to generate its recommendation results. The retriever aims to select a small set of relevant candidates from the entire items with high efficiency; while the ranker, usually more precise but time-consuming, is supposed to further refine the best items from the retrieved candidates. Traditionally, the two components are trained either independently or within a simple cascading pipeline, which is prone to poor collaboration effect. Though some latest works suggested to train retriever and ranker jointly, there still exist many severe limitations: item distribution shift between training and inference, false negative, and misalignment of ranking order. As such, it remains to explore effective collaborations between retriever and ranker.

cs.IR↗

arXiv2023

Structural Imbalance Aware Graph Augmentation Learning

Graph machine learning (GML) has made great progress in node classification, link prediction, graph classification and so on. However, graphs in reality are often structurally imbalanced, that is, only a few hub nodes have a denser local structure and higher influence. The imbalance may compromise the robustness of existing GML models, especially in learning tail nodes. This paper proposes a selective graph augmentation method (SAug) to solve this problem. Firstly, a Pagerank-based sampling strategy is designed to identify hub nodes and tail nodes in the graph. Secondly, a selective augmentation strategy is proposed, which drops the noisy neighbors of hub nodes on one side, and discovers the latent neighbors and generates pseudo neighbors for tail nodes on the other side. It can also alleviate the structural imbalance between two types of nodes. Finally, a GNN model will be retrained on the augmented graph. Extensive experiments demonstrate that SAug can significantly improve the backbone GNNs and achieve superior performance to its competitors of graph augmentation methods and hub/tail aware methods.

cs.LG↗

arXiv2023

Amos-SLAM: An Anti-Dynamics Two-stage SLAM Approach

The traditional Simultaneous Localization And Mapping (SLAM) systems rely on the assumption of a static environment and fail to accurately estimate the system's location when dynamic objects are present in the background. While learning-based dynamic SLAM systems have difficulties in handling unknown moving objects, geometry-based methods have limited success in addressing the residual effects of unidentified dynamic objects on location estimation. To address these issues, we propose an anti-dynamics two-stage SLAM approach. Firstly, the potential motion regions of both prior and non-prior dynamic objects are extracted and pose estimates for dynamic discrimination are quickly obtained using optical flow tracking and model generation methods. Secondly, dynamic points in each frame are removed through dynamic judgment. For non-prior dynamic objects, we present a approach that uses super-pixel extraction and geometric clustering to determine the potential motion regions based on color and geometric information in the image. Evaluations on multiple low and high dynamic sequences in a public RGB-D dataset show that our proposed method outperforms state-of-the-art dynamic SLAM methods.

cs.RO↗

arXiv2023

A Neural Corpus Indexer for Document Retrieval

Current state-of-the-art document retrieval solutions mainly follow an index-retrieve paradigm, where the index is hard to be directly optimized for the final retrieval target. In this paper, we aim to show that an end-to-end deep neural network unifying training and indexing stages can significantly improve the recall performance of traditional methods. To this end, we propose Neural Corpus Indexer (NCI), a sequence-to-sequence network that generates relevant document identifiers directly for a designated query. To optimize the recall performance of NCI, we invent a prefix-aware weight-adaptive decoder architecture, and leverage tailored techniques including query generation, semantic document identifiers, and consistency-based regularization. Empirical studies demonstrated the superiority of NCI on two commonly used academic benchmarks, achieving +21.4% and +16.8% relative enhancement for Recall@1 on NQ320k dataset and R-Precision on TriviaQA dataset, respectively, compared to the best baseline method.

cs.IR↗

arXiv2023

Quantifying proximity-induced superconductivity from first-principles calculations

Proximity induced superconductivity with a clean interface has attracted much attention in recent years. We discuss how the commonly-employed electron tunneling approximation can be hybridized with first-principles calculation to achieve a quantitative characterization starting from the microscopic atomic structure. By using the graphene-Zn heterostructure as an example, we compare this approximated treatment to the full \textit{ab inito} anisotropic Eliashberg formalism. Based on the calculation results, we discuss how superconductivity is affected by the interfacial environment.

cond-mat.supr-con↗

arXiv2023

Async-fork: Mitigating Query Latency Spikes Incurred by the Fork-based Snapshot Mechanism from the OS Level

In-memory key-value stores (IMKVSes) serve many online applications because of their efficiency. To support data backup, popular industrial IMKVSes periodically take a point-in-time snapshot of the in-memory data with the system call fork. However, this mechanism can result in latency spikes for queries arriving during the snapshot period because fork leads the engine into the kernel mode in which the engine is out-of-service for queries. In contrast to existing research focusing on optimizing snapshot algorithms, we optimize the fork operation to address the latency spikes problem from the operating system (OS) level, while keeping the data persistent mechanism in IMKVSes unchanged. Specifically, we first conduct an in-depth study to reveal the impact of the fork operation as well as the optimization techniques on query latency. Based on findings in the study, we propose Async-fork to offload the work of copying the page table from the engine (the parent process) to the child process as copying the page table dominates the execution time of fork. To keep data consistent between the parent and the child, we design the proactive synchronization strategy. Async-fork is implemented in the Linux kernel and deployed into the online Redis database in public clouds. Our experiment results show that compared with the default fork method in OS, Async-fork reduces the tail latency of queries arriving during the snapshot period by 81.76% on an 8GB instance and 99.84% on a 64GB instance.

cs.DB↗

arXiv2023

Emergent Electronic Kagome Lattice in Correlated Charge-Density-Wave State of 1T-TaS$_2$

Quantum materials with tunable correlated and/or topological electronic states, such as the electronic Kagome lattice, provide an ideal platform to study the exotic quantum properties. However, the real-space investigations on the correlated electronic Kagome lattice have been rarely reported. Herein, we report on the electronic Kagome lattice emerging in the correlated charge-density-wave (CDW) state of 1T-TaS$_2$ at ~200 K via variable-temperature scanning tunneling microscopy (VT-STM). This emergent Kagome lattice can be considered a fractional electron-filling superstructure with reduced translational and rotational symmetries, confirmed by STM measurements and density functional theory simulations. The characteristic band structure and density of states of this electronic Kagome lattice are further explored based on theoretical calculations. Our results demonstrate a self-organized electronic Kagome lattice from the correlated CDW state via the effective tuning parameter of temperature and provide a platform to directly explore the interplay of correlated electrons and topological physics.

cond-mat.mtrl-sci↗

arXiv2023

Time-aware Hyperbolic Graph Attention Network for Session-based Recommendation

Session-based Recommendation (SBR) is to predict users' next interested items based on their previous browsing sessions. Existing methods model sessions as graphs or sequences to estimate user interests based on their interacted items to make recommendations. In recent years, graph-based methods have achieved outstanding performance on SBR. However, none of these methods consider temporal information, which is a crucial feature in SBR as it indicates timeliness or currency. Besides, the session graphs exhibit a hierarchical structure and are demonstrated to be suitable in hyperbolic geometry. But few papers design the models in hyperbolic spaces and this direction is still under exploration. In this paper, we propose Time-aware Hyperbolic Graph Attention Network (TA-HGAT) - a novel hyperbolic graph neural network framework to build a session-based recommendation model considering temporal information. More specifically, there are three components in TA-HGAT. First, a hyperbolic projection module transforms the item features into hyperbolic space. Second, the time-aware graph attention module models time intervals between items and the users' current interests. Third, an evolutionary loss at the end of the model provides an accurate prediction of the recommended item based on the given timestamp. TA-HGAT is built in a hyperbolic space to learn the hierarchical structure of session graphs. Experimental results show that the proposed TA-HGAT has the best performance compared to ten baseline models on two real-world datasets.

cs.IR↗

arXiv2022

Layer Imbalance Aware Multiplex Network Embedding

Multiplex network embedding is an effective technique to jointly learn the low-dimensional representations of nodes across network layers. However, the number of edges among layers may vary significantly. This data imbalance will lead to performance degradation especially on the sparse layer due to learning bias and the adverse effects of irrelevant or conflicting data in other layers. In this paper, a Layer Imbalance Aware Multiplex Network Embedding (LIAMNE) method is proposed where the edges in auxiliary layers are under-sampled based on the node similarity in the embedding space of the target layer to achieve balanced edge distribution and to minimize noisy relations that are less relevant to the target layer. Real-world datasets with different degrees of layer imbalance are used for experimentation. The results demonstrate that LIAMNE significantly outperforms several state-of-the-art multiplex network embedding methods in link prediction on the target layer. Meantime, the comprehensive representation of the entire multiplex network is not compromised by the sampling method as evaluated by its performance on the node classification task.

cs.SI↗

arXiv2022

Probing complex stacking in a layered material via electron-nuclear quadrupolar coupling

For layered materials, the interlayer stacking is a critical degree of freedom tuning electronic properties, while its microscopic characterization faces great challenges. The transition-metal dichalcogenide 1T-TaS$_2$ represents a novel example, in which the stacking pattern is not only enriched by the spontaneous occurrence of the intralayer charge density wave, but also recognized as a key to understand the nature of the low-temperature insulating phase. We exploit the $^{33}\rm{S}$ nuclei in a 1T-TaS$_2$ single crystal as sensitive probes of the local stacking pattern via quadrupolar coupling to the electron density distribution nearby, by combining nuclear magnetic resonance (NMR) measurements with the state-of-the-art first-principles electric-field gradient calculations. The applicability of our proposal is analyzed through temperature, magnetic-field, and angle dependent NMR spectra. Systematic simulations of a single 1T-TaS$_2$ layer, bilayers with different stacking patterns, and typical stacking orders in three-dimensional (3D) structures unravel distinct NMR characteristics. Particularly, one 3D structure achieves a quantitative agreement with the experimental spectrum, which clearly rationalizes the coexistence of two types of interfacial environments. Our method may find general applications in the studies of layered materials.

cond-mat.mtrl-sci↗

arXiv2022

Sliding induced multiple polarization states in two-dimensional ferroelectrics

When the atomic layers in a non-centrosymmetric van der Waals structure slide against each other, the interfacial charge transfer results in a reversal of the structures spontaneous polarization. This phenomenon is known as sliding ferroelectricity and it is markedly different from conventional ferroelectric switching mechanisms relying on ion displacement. Here, we present layer dependence as a new dimension to control sliding ferroelectricity. By fabricating 3R MoS2 of various thicknesses into dual-gate field-effect transistors, we obtain anomalous intermediate polarization states in multilayer 3R MoS2. Using results from ab initio density functional theory calculations, we propose a generalized model to describe the ferroelectric switching process in multilayer 3R MoS2 and to explain the formation of these intermediate polarization states. This work reveals the critical roles that layer number and interlayer dipole coupling play in sliding ferroelectricity and presents a new strategy for the design of novel sliding ferroelectric devices.

cond-mat.mtrl-sci↗

arXiv2022

Optical Berry Curvature of Hinge State and Its Detection

We demonstrate that the topological hinge state can possess a nontrivial optical Berry curvature in the nonAbelian formulation of the Berry curvature. It can be readily probed by the circular photogalvanic effect (CPGE), with the light illuminating a specific hinge, and we refer to it as the hinge CPGE. As a concrete example, we calculate the hinge CPGE in ferromagnetic MnBi2nTe3n+1, and find that the hinge CPGE peak structure well reflects the optical Berry curvature of hinge states and the optical sum rule captures the optical Berry curvature between the hinge state and the ground state. Thus, the hinge CPGE provides a promising route towards the optical detection of the hinge state geometrical structure.

cond-mat.mes-hall↗

arXiv2022

Visualizing bulk and edge photocurrent flow in anisotropic Weyl semimetals

Materials that rectify light into current in their bulk are desired for optoelectronic applications. In inversion-breaking Weyl semimetals, bulk photocurrents may arise due to nonlinear optical processes that are enhanced near the Weyl nodes. However, the photoresponse of these materials is commonly studied by scanning photocurrent microscopy (SPCM), which convolves the effects of photocurrent generation and collection. Here, we directly image the photocurrent flow inside the type-II Weyl semimetals WTe2 and TaIrTe4 using high-sensitivity quantum magnetometry with nitrogen-vacancy center spins. We elucidate an unknown mechanism for bulk photocurrent generation termed the anisotropic photothermoelectric effect (APTE), where unequal thermopowers along different crystal axes drive intricate circulations of photocurrent around the photoexcitation. Using simultaneous SPCM and magnetic imaging at the sample's interior and edges, we visualize how the APTE stimulates the long-range photocurrent collected in our Weyl semimetal devices through the Shockley-Ramo theorem. Our results highlight an overlooked, but widely relevant source of current flow and inspire novel photodetectors using homogeneous materials with anisotropy.

cond-mat.mtrl-sci↗