Searcharxiv⌕ Search

arXiv subjects

Zhen Huang

Publications and source records attributed to Zhen Huang.

At least 127 records · Page 7Linked to original sources
arXiv2023

Competition of electronic correlation and reconstruction in La1-xSrxTiO3/SrTiO3 heterostructures

Electronic correlation and reconstruction are two important factors that play a critical role in shaping the magnetic and electronic properties of correlated low-dimensional systems. Here, we report a competition between the electronic correlation and structural reconstruction in La1-xSrxTiO3/SrTiO3 heterostructures by modulating material polarity and interfacial strain, respectively. The heterostructures exhibit a critical thickness (tc) at which a metal-to-insulator transition (MIT) abruptly occurs at certain thickness, accompanied by the coexistence of two- and three-dimensional (2D and 3D) carriers. Intriguingly, the tc exhibits a V-shaped dependence on the doping concentration of Sr, with the smallest tc value at x = 0.5. We attribute this V-shaped dependence to the competition between the electronic reconstruction (modulated by the polarity) and the electronic correlation (modulated by strain), which are borne out by the experimental results, including strain-dependent electronic properties and the evolution of 2D and 3D carriers. Our findings underscore the significance of the interplay between electronic reconstruction and correlation in the realization and utilization of emergent electronic functionalities in low-dimensional correlated systems.

cond-mat.str-el↗
arXiv2023

Stochastic Schrödinger equation approach to real-time dynamics of Anderson-Holstein impurities: an open quantum system perspective

We develop a stochastic Schrödinger equation (SSE) framework to simulate real-time dynamics of Anderson-Holstein (AH) impurities coupled to a continuous fermionic bath. The bath degrees of freedom are incorporated through fluctuating terms determined by exact system-bath correlations, which is derived in an ab initio manner. We show that such an SSE treatment provides a middle ground between numerically expansive microscopic simulations and macroscopic master equations. Computationally, the SSE model enables efficient numerical methods for propagating stochastic trajectories. We demonstrate that this approach not only naturally provides microscopically-detailed information unavailable from reduced models, but also captures effects beyond master equations, thus serves as a promising tool to study open quantum dynamics emerging in physics and chemistry.

quant-ph↗
arXiv2023

DaMSTF: Domain Adversarial Learning Enhanced Meta Self-Training for Domain Adaptation

Self-training emerges as an important research line on domain adaptation. By taking the model's prediction as the pseudo labels of the unlabeled data, self-training bootstraps the model with pseudo instances in the target domain. However, the prediction errors of pseudo labels (label noise) challenge the performance of self-training. To address this problem, previous approaches only use reliable pseudo instances, i.e., pseudo instances with high prediction confidence, to retrain the model. Although these strategies effectively reduce the label noise, they are prone to miss the hard examples. In this paper, we propose a new self-training framework for domain adaptation, namely Domain adversarial learning enhanced Self-Training Framework (DaMSTF). Firstly, DaMSTF involves meta-learning to estimate the importance of each pseudo instance, so as to simultaneously reduce the label noise and preserve hard examples. Secondly, we design a meta constructor for constructing the meta-validation set, which guarantees the effectiveness of the meta-learning module by improving the quality of the meta-validation set. Thirdly, we find that the meta-learning module suffers from the training guidance vanishment and tends to converge to an inferior optimal. To this end, we employ domain adversarial learning as a heuristic neural network initialization method, which can help the meta-learning module converge to a better optimal. Theoretically and experimentally, we demonstrate the effectiveness of the proposed DaMSTF. On the cross-domain sentiment classification task, DaMSTF improves the performance of BERT with an average of nearly 4%.

cs.LG↗
arXiv2023

Meta-Tsallis-Entropy Minimization: A New Self-Training Approach for Domain Adaptation on Text Classification

Text classification is a fundamental task for natural language processing, and adapting text classification models across domains has broad applications. Self-training generates pseudo-examples from the model's predictions and iteratively trains on the pseudo-examples, i.e., minimizes the loss on the source domain and the Gibbs entropy on the target domain. However, Gibbs entropy is sensitive to prediction errors, and thus, self-training tends to fail when the domain shift is large. In this paper, we propose Meta-Tsallis Entropy minimization (MTEM), which applies a meta-learning algorithm to optimize the instance adaptive Tsallis entropy on the target domain. To reduce the computation cost of MTEM, we propose an approximation technique to approximate the Second-order derivation involved in the meta-learning. To efficiently generate pseudo labels, we propose an annealing sampling mechanism for exploring the model's prediction probability. Theoretically, we prove the convergence of the meta-learning algorithm in MTEM and analyze the effectiveness of MTEM in achieving domain adaptation. Experimentally, MTEM improves the adaptation performance of BERT with an average of 4 percent on the benchmark dataset.

cs.CL↗
arXiv2023

Sequential Recommendation with Diffusion Models

Generative models, such as Variational Auto-Encoder (VAE) and Generative Adversarial Network (GAN), have been successfully applied in sequential recommendation. These methods require sampling from probability distributions and adopt auxiliary loss functions to optimize the model, which can capture the uncertainty of user behaviors and alleviate exposure bias. However, existing generative models still suffer from the posterior collapse problem or the model collapse problem, thus limiting their applications in sequential recommendation. To tackle the challenges mentioned above, we leverage a new paradigm of the generative models, i.e., diffusion models, and present sequential recommendation with diffusion models (DiffRec), which can avoid the issues of VAE- and GAN-based models and show better performance. While diffusion models are originally proposed to process continuous image data, we design an additional transition in the forward process together with a transition in the reverse process to enable the processing of the discrete recommendation data. We also design a different noising strategy that only noises the target item instead of the whole sequence, which is more suitable for sequential recommendation. Based on the modified diffusion process, we derive the objective function of our framework using a simplification technique and design a denoise sequential recommender to fulfill the objective function. As the lengthened diffusion steps substantially increase the time complexity, we propose an efficient training strategy and an efficient inference strategy to reduce training and inference cost and improve recommendation diversity. Extensive experiment results on three public benchmark datasets verify the effectiveness of our approach and show that DiffRec outperforms the state-of-the-art sequential recommendation models.

cs.IR↗
arXiv2023

PELE scores: Pelvic X-ray Landmark Detection by Pelvis Extraction and Enhancement

The pelvis, the lower part of the trunk, supports and balances the trunk. Landmark detection from a pelvic X-ray (PXR) facilitates downstream analysis and computer-assisted diagnosis and treatment of pelvic diseases. Although PXRs have the advantages of low radiation and reduced cost compared to computed tomography (CT) images, their 2D pelvis-tissue superposition of 3D structures confuses clinical decision-making. In this paper, we propose a PELvis Extraction (PELE) module that utilizes 3D prior anatomical knowledge in CT to guide and well isolate the pelvis from PXRs, thereby eliminating the influence of soft tissue. We conduct an extensive evaluation based on two public datasets and one private dataset, totaling 850 PXRs. The experimental results show that the proposed PELE module significantly improves the accuracy of PXRs landmark detection and achieves state-of-the-art performances in several benchmark metrics, thus better serving downstream tasks.

eess.IV↗
arXiv2023

DiffuseExpand: Expanding dataset for 2D medical image segmentation using diffusion models

Dataset expansion can effectively alleviate the problem of data scarcity for medical image segmentation, due to privacy concerns and labeling difficulties. However, existing expansion algorithms still face great challenges due to their inability of guaranteeing the diversity of synthesized images with paired segmentation masks. In recent years, Diffusion Probabilistic Models (DPMs) have shown powerful image synthesis performance, even better than Generative Adversarial Networks. Based on this insight, we propose an approach called DiffuseExpand for expanding datasets for 2D medical image segmentation using DPM, which first samples a variety of masks from Gaussian noise to ensure the diversity, and then synthesizes images to ensure the alignment of images and masks. After that, DiffuseExpand chooses high-quality samples to further enhance the effectiveness of data expansion. Our comparison and ablation experiments on COVID-19 and CGMH Pelvis datasets demonstrate the effectiveness of DiffuseExpand. Our code is released at https://github.com/shaoshitong/DiffuseExpand.

eess.IV↗
arXiv2023

Teaching What You Should Teach: A Data-Based Distillation Method

In real teaching scenarios, an excellent teacher always teaches what he (or she) is good at but the student is not. This gives the student the best assistance in making up for his (or her) weaknesses and becoming a good one overall. Enlightened by this, we introduce the "Teaching what you Should Teach" strategy into a knowledge distillation framework, and propose a data-based distillation method named "TST" that searches for desirable augmented samples to assist in distilling more efficiently and rationally. To be specific, we design a neural network-based data augmentation module with priori bias, which assists in finding what meets the teacher's strengths but the student's weaknesses, by learning magnitudes and probabilities to generate suitable data samples. By training the data augmentation module and the generalized distillation paradigm in turn, a student model is learned with excellent generalization ability. To verify the effectiveness of our method, we conducted extensive comparative experiments on object recognition, detection, and segmentation tasks. The results on the CIFAR-10, ImageNet-1k, MS-COCO, and Cityscapes datasets demonstrate that our method achieves state-of-the-art performance on almost all teacher-student pairs. Furthermore, we conduct visualization studies to explore what magnitudes and probabilities are needed for the distillation process.

cs.CV↗
arXiv2023

Multivariate Symmetry: Distribution-Free Testing via Optimal Transport

The sign test (Arbuthnott, 1710) and the Wilcoxon signed-rank test (Wilcoxon, 1945) are among the first examples of a nonparametric test. These procedures -- based on signs, (absolute) ranks and signed-ranks -- yield distribution-free tests for symmetry in one-dimension. In this paper we propose a novel and unified framework for distribution-free testing of multivariate symmetry (that includes central symmetry, sign symmetry, spherical symmetry, etc.) based on the theory of optimal transport. Our approach leads to notions of distribution-free generalized multivariate signs, ranks and signed-ranks. As a consequence, we develop analogues of the sign and Wilcoxon signed-rank tests that share many of the appealing properties of their one-dimensional counterparts. In particular, the proposed tests are exactly distribution-free in finite samples with an asymptotic normal limit, and adapt to various notions of multivariate symmetry. We study the consistency of the proposed tests and their behavior under local alternatives, and show that the proposed generalized Wilcoxon signed-rank (GWSR) test is particularly powerful against location shift alternatives. We show that in a large class of such models, our GWSR test suffers from no loss in (asymptotic) efficiency, when compared to Hotelling's $T^2$ test, despite being nonparametric and exactly distribution-free. An appropriately score transformed version of the GWSR statistic leads to a locally asymptotically optimal test. Further, our method can be readily used to construct distribution-free confidence sets for the center of symmetry.

stat.ME↗
arXiv2023

Incorporating Structured Sentences with Time-enhanced BERT for Fully-inductive Temporal Relation Prediction

Temporal relation prediction in incomplete temporal knowledge graphs (TKGs) is a popular temporal knowledge graph completion (TKGC) problem in both transductive and inductive settings. Traditional embedding-based TKGC models (TKGE) rely on structured connections and can only handle a fixed set of entities, i.e., the transductive setting. In the inductive setting where test TKGs contain emerging entities, the latest methods are based on symbolic rules or pre-trained language models (PLMs). However, they suffer from being inflexible and not time-specific, respectively. In this work, we extend the fully-inductive setting, where entities in the training and test sets are totally disjoint, into TKGs and take a further step towards a more flexible and time-sensitive temporal relation prediction approach SST-BERT, incorporating Structured Sentences with Time-enhanced BERT. Our model can obtain the entity history and implicitly learn rules in the semantic space by encoding structured sentences, solving the problem of inflexibility. We propose to use a time masking MLM task to pre-train BERT in a corpus rich in temporal tokens specially generated for TKGs, enhancing the time sensitivity of SST-BERT. To compute the probability of occurrence of a target quadruple, we aggregate all its structured sentences from both temporal and semantic perspectives into a score. Experiments on the transductive datasets and newly generated fully-inductive benchmarks show that SST-BERT successfully improves over state-of-the-art baselines.

cs.CL↗
arXiv2023

Meta-Learning Based Knowledge Extrapolation for Temporal Knowledge Graph

In the last few years, the solution to Knowledge Graph (KG) completion via learning embeddings of entities and relations has attracted a surge of interest. Temporal KGs(TKGs) extend traditional Knowledge Graphs (KGs) by associating static triples with timestamps forming quadruples. Different from KGs and TKGs in the transductive setting, constantly emerging entities and relations in incomplete TKGs create demand to predict missing facts with unseen components, which is the extrapolation setting. Traditional temporal knowledge graph embedding (TKGE) methods are limited in the extrapolation setting since they are trained within a fixed set of components. In this paper, we propose a Meta-Learning based Temporal Knowledge Graph Extrapolation (MTKGE) model, which is trained on link prediction tasks sampled from the existing TKGs and tested in the emerging TKGs with unseen entities and relations. Specifically, we meta-train a GNN framework that captures relative position patterns and temporal sequence patterns between relations. The learned embeddings of patterns can be transferred to embed unseen components. Experimental results on two different TKG extrapolation datasets show that MTKGE consistently outperforms both the existing state-of-the-art models for knowledge graph extrapolation and specifically adapted KGE and TKGE baselines.

cs.AI↗
arXiv2022

Toward Real-world Single Image Deraining: A New Benchmark and Beyond

Single image deraining (SID) in real scenarios attracts increasing attention in recent years. Due to the difficulty in obtaining real-world rainy/clean image pairs, previous real datasets suffer from low-resolution images, homogeneous rain streaks, limited background variation, and even misalignment of image pairs, resulting in incomprehensive evaluation of SID methods. To address these issues, we establish a new high-quality dataset named RealRain-1k, consisting of $1,120$ high-resolution paired clean and rainy images with low- and high-density rain streaks, respectively. Images in RealRain-1k are automatically generated from a large number of real-world rainy video clips through a simple yet effective rain density-controllable filtering method, and have good properties of high image resolution, background diversity, rain streaks variety, and strict spatial alignment. RealRain-1k also provides abundant rain streak layers as a byproduct, enabling us to build a large-scale synthetic dataset named SynRain-13k by pasting the rain streak layers on abundant natural images. Based on them and existing datasets, we benchmark more than 10 representative SID methods on three tracks: (1) fully supervised learning on RealRain-1k, (2) domain generalization to real datasets, and (3) syn-to-real transfer learning. The experimental results (1) show the difference of representative methods in image restoration performance and model complexity, (2) validate the significance of the proposed datasets for model generalization, and (3) provide useful insights on the superiority of learning from diverse domains and shed lights on the future research on real-world SID. The datasets will be released at https://github.com/hiker-lw/RealRain-1k

cs.CV↗
arXiv2022

R$^2$F: A General Retrieval, Reading and Fusion Framework for Document-level Natural Language Inference

Document-level natural language inference (DOCNLI) is a new challenging task in natural language processing, aiming at judging the entailment relationship between a pair of hypothesis and premise documents. Current datasets and baselines largely follow sentence-level settings, but fail to address the issues raised by longer documents. In this paper, we establish a general solution, named Retrieval, Reading and Fusion (R2F) framework, and a new setting, by analyzing the main challenges of DOCNLI: interpretability, long-range dependency, and cross-sentence inference. The basic idea of the framework is to simplify document-level task into a set of sentence-level tasks, and improve both performance and interpretability with the power of evidence. For each hypothesis sentence, the framework retrieves evidence sentences from the premise, and reads to estimate its credibility. Then the sentence-level results are fused to judge the relationship between the documents. For the setting, we contribute complementary evidence and entailment label annotation on hypothesis sentences, for interpretability study. Our experimental results show that R2F framework can obtain state-of-the-art performance and is robust for diverse evidence retrieval methods. Moreover, it can give more interpretable prediction results. Our model and code are released at https://github.com/phoenixsecularbird/R2F.

cs.CL↗
arXiv2022

A Treatise On FST Lattice Based MMI Training

Maximum mutual information (MMI) has become one of the two de facto methods for sequence-level training of speech recognition acoustic models. This paper aims to isolate, identify and bring forward the implicit modelling decisions induced by the design implementation of standard finite state transducer (FST) lattice based MMI training framework. The paper particularly investigates the necessity to maintain a preselected numerator alignment and raises the importance of determinizing FST denominator lattices on the fly. The efficacy of employing on the fly FST lattice determinization is mathematically shown to guarantee discrimination at the hypothesis level and is empirically shown through training deep CNN models on a 18K hours Mandarin dataset and on a 2.8K hours English dataset. On assistant and dictation tasks, the approach achieves between 2.3-4.6% relative WER reduction (WERR) over the standard FST lattice based approach.

cs.LG↗
arXiv2022

A Kernel Measure of Dissimilarity between $M$ Distributions

Given $M \geq 2$ distributions defined on a general measurable space, we introduce a nonparametric (kernel) measure of multi-sample dissimilarity (KMD) -- a parameter that quantifies the difference between the $M$ distributions. The population KMD, which takes values between 0 and 1, is 0 if and only if all the $M$ distributions are the same, and 1 if and only if all the distributions are mutually singular. Moreover, KMD possesses many properties commonly associated with $f$-divergences such as the data processing inequality and invariance under bijective transformations. The sample estimate of KMD, based on independent observations from the $M$ distributions, can be computed in near linear time (up to logarithmic factors) using $k$-nearest neighbor graphs (for $k \ge 1$ fixed). We develop an easily implementable test for the equality of $M$ distributions based on the sample KMD that is consistent against all alternatives where at least two distributions are not equal. We prove central limit theorems for the sample KMD, and provide a complete characterization of the asymptotic power of the test, as well as its detection threshold. The usefulness of our measure is demonstrated via real and synthetic data examples; our method is also implemented in an R package.

math.ST↗
arXiv2022

AstBERT: Enabling Language Model for Financial Code Understanding with Abstract Syntax Trees

Using the pre-trained language models to understand source codes has attracted increasing attention from financial institutions owing to the great potential to uncover financial risks. However, there are several challenges in applying these language models to solve programming language-related problems directly. For instance, the shift of domain knowledge between natural language (NL) and programming language (PL) requires understanding the semantic and syntactic information from the data from different perspectives. To this end, we propose the AstBERT model, a pre-trained PL model aiming to better understand the financial codes using the abstract syntax tree (AST). Specifically, we collect a sheer number of source codes (both Java and Python) from the Alipay code repository and incorporate both syntactic and semantic code knowledge into our model through the help of code parsers, in which AST information of the source codes can be interpreted and integrated. We evaluate the performance of the proposed model on three tasks, including code question answering, code clone detection and code refinement. Experiment results show that our AstBERT achieves promising performance on three different downstream tasks.

cs.AI↗
arXiv2022

Robust analytic continuation of Green's functions via projection, pole estimation, and semidefinite relaxation

Green's functions of fermions are described by matrix-valued Herglotz-Nevanlinna functions. Since analytic continuation is fundamentally an ill-posed problem, the causal space described by the matrix-valued Herglotz-Nevanlinna structure can be instrumental in improving the accuracy and in enhancing the robustness with respect to noise. We demonstrate a three-pronged procedure for robust analytic continuation called PES: (1) Projection of data to the causal space. (2) Estimation of pole locations. (3) Semidefinite relaxation within the causal space. We compare the performance of PES with the recently developed Nevanlinna and Carathéodory continuation methods and find that PES is more robust in the presence of noise and does not require the usage of extended precision arithmetics. We also demonstrate that a causal projection improves the performance of the Nevanlinna and Carathéodory methods. The PES method is generalized to bosonic response functions, for which the Nevanlinna and Carathéodory continuation methods have not yet been developed. It is particularly useful for studying spectra with sharp features, as they occur in the study of molecules and band structures in solids.

cond-mat.str-el↗
arXiv2022

IMCI: Integrate Multi-view Contextual Information for Fact Extraction and Verification

With the rapid development of automatic fake news detection technology, fact extraction and verification (FEVER) has been attracting more attention. The task aims to extract the most related fact evidences from millions of open-domain Wikipedia documents and then verify the credibility of corresponding claims. Although several strong models have been proposed for the task and they have made great progress, we argue that they fail to utilize multi-view contextual information and thus cannot obtain better performance. In this paper, we propose to integrate multi-view contextual information (IMCI) for fact extraction and verification. For each evidence sentence, we define two kinds of context, i.e. intra-document context and inter-document context}. Intra-document context consists of the document title and all the other sentences from the same document. Inter-document context consists of all other evidences which may come from different documents. Then we integrate the multi-view contextual information to encode the evidence sentences to handle the task. Our experimental results on FEVER 1.0 shared task show that our IMCI framework makes great progress on both fact extraction and verification, and achieves state-of-the-art performance with a winning FEVER score of 72.97% and label accuracy of 75.84% on the online blind test set. We also conduct ablation study to detect the impact of multi-view contextual information. Our codes will be released at https://github.com/phoenixsecularbird/IMCI.

cs.CL↗