arXiv subjects

Zhenyu Zhang

Publications and source records attributed to Zhenyu Zhang.

At least 235 records · Page 13Linked to original sources

arXiv2023

Improving Distantly-Supervised Named Entity Recognition with Self-Collaborative Denoising Learning

Distantly supervised named entity recognition (DS-NER) efficiently reduces labor costs but meanwhile intrinsically suffers from the label noise due to the strong assumption of distant supervision. Typically, the wrongly labeled instances comprise numbers of incomplete and inaccurate annotation noise, while most prior denoising works are only concerned with one kind of noise and fail to fully explore useful information in the whole training set. To address this issue, we propose a robust learning paradigm named Self-Collaborative Denoising Learning (SCDL), which jointly trains two teacher-student networks in a mutually-beneficial manner to iteratively perform noisy label refinery. Each network is designed to exploit reliable labels via self denoising, and two networks communicate with each other to explore unreliable annotations by collaborative denoising. Extensive experimental results on five real-world datasets demonstrate that SCDL is superior to state-of-the-art DS-NER denoising methods.

cs.CL↗

arXiv2023

Two-dimensional superconductors with intrinsic p-wave pairing or nontrivial band topology

Over the past fifteen years, tremendous efforts have been devoted to realizing topological superconductivity in realistic materials and systems, predominately propelled by their promising application potentials in fault-tolerant quantum information processing. In this article, we attempt to give an overview on some of the main developments in this field, focusing in particular on two-dimensional crystalline superconductors that possess either intrinsic p-wave pairing or nontrivial band topology. We first classify the three different conceptual schemes to achieve topological superconductor (TSC), enabled by real-space superconducting proximity effect, reciprocal-space superconducting proximity effect, and intrinsic TSC. Whereas the first scheme has so far been most extensively explored, the subtle difference between the other two remains to be fully substantiated. We then move on to candidate intrinsic or p-wave superconductors, including Sr2RuO4, UTe2, Pb3Bi, and graphene-based systems. For TSC systems that rely on proximity effects, the emphases are mainly on the coexistence of superconductivity and nontrivial band topology, as exemplified by transition metal dichalcogenides, cobalt pnictides, and stanene, all in monolayer or few-layer regime. The review completes with discussions on the three dominant tuning schemes of strain, gating, and ferroelectricity in acquiring one or both essential ingredients of the TSC, and optimizations of such tuning capabilities may prove to be decisive in our drive towards braiding of Majorana zero modes and demonstration of topological qubits.

cond-mat.supr-con↗

arXiv2023

Realization of multiple charge density waves in NbTe2 at the monolayer limit

Abstract: Layered transition-metal dichalcogenides (TMDCs) down to the monolayer (ML) limit provide a fertile platform for exploring charge-density waves (CDWs). Though bulk NbTe2 is known to harbor a single axis 3*1 CDW coexisting with non-trivial quantum properties, the scenario in the ML limit is still experimentally unknown. In this study, we unveil the richness of the CDW phases in ML NbTe2, where not only the theoretically predicted 4*4 and 4*1 phases, but also two unexpected sqrt(28)*sqrt(28) and sqrt(19)*sqrt(19) phases, can be realized. For such a complex CDW system, we establish an exhaustive growth phase diagram via systematic efforts in the material synthesis and scanning tunneling microscope characterization. Moreover, we report that the energetically stable phase is the larger scale order (sqrt(19)*sqrt(19)), which is surprisingly in contradiction to the prior prediction (4*4). These findings are confirmed using two different kinetic pathways, i.e., direct growth at proper growth temperatures (T), and low-T growth followed by high-T annealing. Our results provide a comprehensive diagram of the "zoo" of CDW orders in ML 1T-NbTe2 for the first time and offer a new material platform for studying novel quantum phases in the 2D limit.

cond-mat.mtrl-sci↗

arXiv2023

Shadows of Kerr black holes with a Gaussian-distributed plasma in the polar direction

In this work, we study the shadow of Kerr black hole surrounded by an axisymmetric plasma, whose density takes a Gaussian distribution in the polar direction. Along the radial direction, we consider two models: in model A the density of the plasma decays in a power law; in model B the density obeys a logarithmic Gaussian distribution. Using the numerical backward ray-tracing method, we find that the size of the shadow is sensitive to the inclination angle of the observer due to the polar distribution of the density of the plasma. In particular, we pay special attention to the model B and investigate the influence of the radial position of maximum density, the decay rate of the density towards the event horizon and the opening angle of the plasma on the shape and size of the Kerr black hole shadow. The effects of the plasmas studied in this work can be qualitatively explained by taking the plasmas as convex lenses with the refractive index being less than $1$.

gr-qc↗

arXiv2022

Electromagnetic Dalitz Decays of $D_{(s)}^\ast$ Mesons

Rare electromagnetic decays of charmed mesons are useful laboratories to explore the structure of hadronic states and the interactions between photon and charmed mesons, to test the chiral perturbation theory in flavor sector and to search for new physics including dark photons. In this paper, we calculate the relative branching ratios of electromagnetic Dalitz decays $D_{(s)}^\ast\to D_{(s)}\ell^+\ell^-$ to their corresponding radiative decays $D_{(s)}^\ast\to D_{(s)}γ$, the dileptonic invariant mass spectra and the leptonic angular distributions with transition form factor in Vector-Meson Dominance model, where $D_{(s)}^\ast$ represents $D^\ast(2007)^0$, $D^\ast(2010)^\pm$, $D^\ast(2640)^\pm$, $D_s^{\ast\pm}$, $D_{s1}^\ast(2700)^\pm$ and $D_{s1}^\ast(2860)^\pm$.

hep-ph↗

arXiv2022

Towards Generalized Open Information Extraction

Open Information Extraction (OpenIE) facilitates the open-domain discovery of textual facts. However, the prevailing solutions evaluate OpenIE models on in-domain test sets aside from the training corpus, which certainly violates the initial task principle of domain-independence. In this paper, we propose to advance OpenIE towards a more realistic scenario: generalizing over unseen target domains with different data distributions from the source training domains, termed Generalized OpenIE. For this purpose, we first introduce GLOBE, a large-scale human-annotated multi-domain OpenIE benchmark, to examine the robustness of recent OpenIE models to domain shifts, and the relative performance degradation of up to 70% implies the challenges of generalized OpenIE. Then, we propose DragonIE, which explores a minimalist graph expression of textual fact: directed acyclic graph, to improve the OpenIE generalization. Extensive experiments demonstrate that DragonIE beats the previous methods in both in-domain and out-of-domain settings by as much as 6.0% in F1 score absolutely, but there is still ample room for improvement.

cs.CL↗

arXiv2022

DesNet: Decomposed Scale-Consistent Network for Unsupervised Depth Completion

Unsupervised depth completion aims to recover dense depth from the sparse one without using the ground-truth annotation. Although depth measurement obtained from LiDAR is usually sparse, it contains valid and real distance information, i.e., scale-consistent absolute depth values. Meanwhile, scale-agnostic counterparts seek to estimate relative depth and have achieved impressive performance. To leverage both the inherent characteristics, we thus suggest to model scale-consistent depth upon unsupervised scale-agnostic frameworks. Specifically, we propose the decomposed scale-consistent learning (DSCL) strategy, which disintegrates the absolute depth into relative depth prediction and global scale estimation, contributing to individual learning benefits. But unfortunately, most existing unsupervised scale-agnostic frameworks heavily suffer from depth holes due to the extremely sparse depth input and weak supervised signal. To tackle this issue, we introduce the global depth guidance (GDG) module, which attentively propagates dense depth reference into the sparse target via novel dense-to-sparse attention. Extensive experiments show the superiority of our method on outdoor KITTI benchmark, ranking 1st and outperforming the best KBNet more than 12% in RMSE. In addition, our approach achieves state-of-the-art performance on indoor NYUv2 dataset.

cs.CV↗

arXiv2022

QuanGCN: Noise-Adaptive Training for Robust Quantum Graph Convolutional Networks

Quantum neural networks (QNNs), an interdisciplinary field of quantum computing and machine learning, have attracted tremendous research interests due to the specific quantum advantages. Despite lots of efforts developed in computer vision domain, one has not fully explored QNNs for the real-world graph property classification and evaluated them in the quantum device. To bridge the gap, we propose quantum graph convolutional networks (QuanGCN), which learns the local message passing among nodes with the sequence of crossing-gate quantum operations. To mitigate the inherent noises from modern quantum devices, we apply sparse constraint to sparsify the nodes' connections and relieve the error rate of quantum gates, and use skip connection to augment the quantum outputs with original node features to improve robustness. The experimental results show that our QuanGCN is functionally comparable or even superior than the classical algorithms on several benchmark graph datasets. The comprehensive evaluations in both simulator and real quantum machines demonstrate the applicability of QuanGCN to the future graph analysis problem.

quant-ph↗

arXiv2022

ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding

Recent years have witnessed the rise and success of pre-training techniques in visually-rich document understanding. However, most existing methods lack the systematic mining and utilization of layout-centered knowledge, leading to sub-optimal performances. In this paper, we propose ERNIE-Layout, a novel document pre-training solution with layout knowledge enhancement in the whole workflow, to learn better representations that combine the features from text, layout, and image. Specifically, we first rearrange input sequences in the serialization stage, and then present a correlative pre-training task, reading order prediction, to learn the proper reading order of documents. To improve the layout awareness of the model, we integrate a spatial-aware disentangled attention into the multi-modal transformer and a replaced regions prediction task into the pre-training phase. Experimental results show that ERNIE-Layout achieves superior performance on various downstream tasks, setting new state-of-the-art on key information extraction, document image classification, and document question answering datasets. The code and models are publicly available at http://github.com/PaddlePaddle/PaddleNLP/tree/develop/model_zoo/ernie-layout.

cs.CL↗

arXiv2022

Image of Kerr-Melvin black hole with thin accretion disk

In this present work, we study the observational appearance of Kerr-Melvin black hole (KMBH) illuminated by an accretion disk. The accretion disk is assumed to be located on the equatorial plane and be thin both geometrically and optically. Considering the fact that outside the innermost stable circular orbit (ISCO) the accretion flow moves in prograde or retrograde circular orbit and falls towards the horizon along plunging orbit inside the ISCO, we develop the numerical backward ray-tracing method and obtain the images of KMBH accompanying with the accretion disk for various black hole spins, strengths of magnetic fields and inclination angles of observers. We present the intensity distribution horizontally and longitudinally and show the profiles of the red-shift for the direct and lensed images. Our study suggests that the inner shadow and critical curves can be used to estimate the magnetic field around a black hole without degeneration.

gr-qc↗

arXiv2022

hdtg: An R package for high-dimensional truncated normal simulation

Simulating from the multivariate truncated normal distribution (MTN) is required in various statistical applications yet remains challenging in high dimensions. Currently available algorithms and their implementations often fail when the number of parameters exceeds a few hundred. To provide a general computational tool to efficiently sample from high-dimensional MTNs, we introduce the hdtg package that implements two state-of-the-art simulation algorithms: harmonic Hamiltonian Monte Carlo (harmonic-HMC) and zigzag Hamiltonian Monte Carlo (Zigzag-HMC). Both algorithms exploit analytical solutions of the Hamiltonian dynamics under a quadratic potential energy with hard boundary constraints, leading to rejection-free methods. We compare their efficiencies against another state-of-the-art algorithm for MTN simulation, the minimax tilting accept-reject sampler (MET). The run-time of these three approaches heavily depends on the underlying multivariate normal correlation structure. Zigzag-HMC and harmonic-HMC both achieve 100 effective samples within 3,600 seconds across all tests with dimension ranging from 100 to 1,600, while MET has difficulty in several high-dimensional examples. We provide guidance on how to choose an appropriate method for a given situation and illustrate the usage of hdtg.

stat.CO↗

arXiv2022

An Efficient End-to-End Transformer with Progressive Tri-modal Attention for Multi-modal Emotion Recognition

Recent works on multi-modal emotion recognition move towards end-to-end models, which can extract the task-specific features supervised by the target task compared with the two-phase pipeline. However, previous methods only model the feature interactions between the textual and either acoustic and visual modalities, ignoring capturing the feature interactions between the acoustic and visual modalities. In this paper, we propose the multi-modal end-to-end transformer (ME2ET), which can effectively model the tri-modal features interaction among the textual, acoustic, and visual modalities at the low-level and high-level. At the low-level, we propose the progressive tri-modal attention, which can model the tri-modal feature interactions by adopting a two-pass strategy and can further leverage such interactions to significantly reduce the computation and memory complexity through reducing the input token length. At the high-level, we introduce the tri-modal feature fusion layer to explicitly aggregate the semantic representations of three modalities. The experimental results on the CMU-MOSEI and IEMOCAP datasets show that ME2ET achieves the state-of-the-art performance. The further in-depth analysis demonstrates the effectiveness, efficiency, and interpretability of the proposed progressive tri-modal attention, which can help our model to achieve better performance while significantly reducing the computation and memory cost. Our code will be publicly available.

cs.CL↗

arXiv2022

Accelerating Bayesian inference of dependency between complex biological traits

Inferring dependencies between complex biological traits while accounting for evolutionary relationships between specimens is of great scientific interest yet remains infeasible when trait and specimen counts grow large. The state-of-the-art approach uses a phylogenetic multivariate probit model to accommodate binary and continuous traits via a latent variable framework, and utilizes an efficient bouncy particle sampler (BPS) to tackle the computational bottleneck -- integrating many latent variables from a high-dimensional truncated normal distribution. This approach breaks down as the number of specimens grows and fails to reliably characterize conditional dependencies between traits. Here, we propose an inference pipeline for phylogenetic probit models that greatly outperforms BPS. The novelty lies in 1) a combination of the recent Zigzag Hamiltonian Monte Carlo (Zigzag-HMC) with linear-time gradient evaluations and 2) a joint sampling scheme for highly correlated latent variables and correlation matrix elements. In an application exploring HIV-1 evolution from 535 viruses, the inference requires joint sampling from an 11,235-dimensional truncated normal and a 24-dimensional covariance matrix. Our method yields a 5-fold speedup compared to BPS and makes it possible to learn partial correlations between candidate viral mutations and virulence. Computational speedup now enables us to tackle even larger problems: we study the evolution of influenza H1N1 glycosylations on around 900 viruses. For broader applicability, we extend the phylogenetic probit model to incorporate categorical traits, and demonstrate its use to study Aquilegia flower and pollinator co-evolution.

stat.ME↗

arXiv2022

NTIRE 2021 Challenge on Quality Enhancement of Compressed Video: Methods and Results

This paper reviews the first NTIRE challenge on quality enhancement of compressed video, with a focus on the proposed methods and results. In this challenge, the new Large-scale Diverse Video (LDV) dataset is employed. The challenge has three tracks. Tracks 1 and 2 aim at enhancing the videos compressed by HEVC at a fixed QP, while Track 3 is designed for enhancing the videos compressed by x265 at a fixed bit-rate. Besides, the quality enhancement of Tracks 1 and 3 targets at improving the fidelity (PSNR), and Track 2 targets at enhancing the perceptual quality. The three tracks totally attract 482 registrations. In the test phase, 12 teams, 8 teams and 11 teams submitted the final results of Tracks 1, 2 and 3, respectively. The proposed methods and solutions gauge the state-of-the-art of video quality enhancement. The homepage of the challenge: https://github.com/RenYang-home/NTIRE21_VEnh

eess.IV↗

arXiv2022

Discover the GellMann-Okubo formula with machine learning

Machine learning is a novel and powerful technology and has been widely used in various science topics. We demonstrate a machine-learning based approach built by a set of general metrics and rules inspired by physics. Taking advantages of physical constraints, such as dimension identity, symmetry and generalization, we succeed to rediscover the GellMann Okubo formula using a technique of symbolic regression. This approach can effectively find explicit solutions among user-defined observable, and easily extend to study on exotic hadron spectrum.

hep-ph↗

arXiv2022

Layout-Aware Information Extraction for Document-Grounded Dialogue: Dataset, Method and Demonstration

Building document-grounded dialogue systems have received growing interest as documents convey a wealth of human knowledge and commonly exist in enterprises. Wherein, how to comprehend and retrieve information from documents is a challenging research problem. Previous work ignores the visual property of documents and treats them as plain text, resulting in incomplete modality. In this paper, we propose a Layout-aware document-level Information Extraction dataset, LIE, to facilitate the study of extracting both structural and semantic knowledge from visually rich documents (VRDs), so as to generate accurate responses in dialogue systems. LIE contains 62k annotations of three extraction tasks from 4,061 pages in product and official documents, becoming the largest VRD-based information extraction dataset to the best of our knowledge. We also develop benchmark methods that extend the token-based language model to consider layout features like humans. Empirical results show that layout is critical for VRD-based extraction, and system demonstration also verifies that the extracted knowledge can help locate the answers that users care about.

cs.CL↗

arXiv2022

RigNet: Repetitive Image Guided Network for Depth Completion

Depth completion deals with the problem of recovering dense depth maps from sparse ones, where color images are often used to facilitate this task. Recent approaches mainly focus on image guided learning frameworks to predict dense depth. However, blurry guidance in the image and unclear structure in the depth still impede the performance of the image guided frameworks. To tackle these problems, we explore a repetitive design in our image guided network to gradually and sufficiently recover depth values. Specifically, the repetition is embodied in both the image guidance branch and depth generation branch. In the former branch, we design a repetitive hourglass network to extract discriminative image features of complex environments, which can provide powerful contextual instruction for depth prediction. In the latter branch, we introduce a repetitive guidance module based on dynamic convolution, in which an efficient convolution factorization is proposed to simultaneously reduce its complexity and progressively model high-frequency structures. Extensive experiments show that our method achieves superior or competitive results on KITTI benchmark and NYUv2 dataset.

cs.CV↗

arXiv2022

Multi-Modal Masked Pre-Training for Monocular Panoramic Depth Completion

In this paper, we formulate a potentially valuable panoramic depth completion (PDC) task as panoramic 3D cameras often produce 360° depth with missing data in complex scenes. Its goal is to recover dense panoramic depths from raw sparse ones and panoramic RGB images. To deal with the PDC task, we train a deep network that takes both depth and image as inputs for the dense panoramic depth recovery. However, it needs to face a challenging optimization problem of the network parameters due to its non-convex objective function. To address this problem, we propose a simple yet effective approach termed M{^3}PT: multi-modal masked pre-training. Specifically, during pre-training, we simultaneously cover up patches of the panoramic RGB image and sparse depth by shared random mask, then reconstruct the sparse depth in the masked regions. To our best knowledge, it is the first time that we show the effectiveness of masked pre-training in a multi-modal vision task, instead of the single-modal task resolved by masked autoencoders (MAE). Different from MAE where fine-tuning completely discards the decoder part of pre-training, there is no architectural difference between the pre-training and fine-tuning stages in our M$^{3}$PT as they only differ in the prediction density, which potentially makes the transfer learning more convenient and effective. Extensive experiments verify the effectiveness of M{^3}PT on three panoramic datasets. Notably, we improve the state-of-the-art baselines by averagely 26.2% in RMSE, 51.7% in MRE, 49.7% in MAE, and 37.5% in RMSElog on three benchmark datasets.

cs.CV↗