arXiv subjects

Zhen Bi

Publications and source records attributed to Zhen Bi.

At least 55 records · Page 3Linked to original sources

arXiv2024

Topological Phases and Phase Transitions with Dipolar Symmetry Breaking

Systems with dipole moment conservation have been of recent interest, as they realize both novel quantum dynamics and exotic ground state phases. In this work, we study some generic properties of 1-D and 2-D dipole-conserving fermionic models at integer fillings. We find that a dipolar symmetry-breaking phase can result in a mean-field band insulator whose topological indices can strongly affect the low-energy physics of the dipolar Goldstone modes. We study the 2-D topological phase transition of the mean-field ground states in the presence of the Goldstone modes. The critical theory resembles the 2+1d quantum electrodynamics coupled to massless Dirac fermions with some crucial differences and shows a novel quantum critical point featuring a nontrivial dynamical exponent. We also discuss the analogous case of 1-D dipole-conserving models and the role of topological invariants.

cond-mat.str-el↗

arXiv2024

CodeKGC: Code Language Model for Generative Knowledge Graph Construction

Current generative knowledge graph construction approaches usually fail to capture structural knowledge by simply flattening natural language into serialized texts or a specification language. However, large generative language model trained on structured data such as code has demonstrated impressive capability in understanding natural language for structural prediction and reasoning tasks. Intuitively, we address the task of generative knowledge graph construction with code language model: given a code-format natural language input, the target is to generate triples which can be represented as code completion tasks. Specifically, we develop schema-aware prompts that effectively utilize the semantic structure within the knowledge graph. As code inherently possesses structure, such as class and function definitions, it serves as a useful model for prior semantic structural knowledge. Furthermore, we employ a rationale-enhanced generation method to boost the performance. Rationales provide intermediate steps, thereby improving knowledge extraction abilities. Experimental results indicate that the proposed approach can obtain better performance on benchmark datasets compared with baselines. Code and datasets are available in https://github.com/zjunlp/DeepKE/tree/main/example/llm.

cs.CL↗

arXiv2024

Maximally Localized Wannier Orbitals, Interaction Models and Fractional Quantum Anomalous Hall Effect in Twisted Bilayer MoTe2

We investigate the moiré band structures and the strong correlation effects in twisted bilayer MoTe$_2$ for a wide range of twist angles, employing a combination of various techniques. Using large-scale first principles calculations, we pinpoint realistic continuum modeling parameters, subsequently deriving the maximally localized Wannier functions for the top three moiré bands. Simplifying our model with reasonable assumptions, we obtain a minimal two-band model, encompassing Coulomb repulsion, correlated hopping, and spin exchange. Our minimal interaction models pave the way for further exploration of the rich many-body physics in twisted MoTe$_2$. Furthermore, we explore the phase diagrams of the system through Hartree-Fock approximation and exact diagonalization. Our two-band exact diagonalization analysis underscores significant band-mixing effects in this system, which enlarge the optimal twist angle for fractional quantum anomalous Hall states.

cond-mat.str-el↗

arXiv2023

When Do Program-of-Thoughts Work for Reasoning?

In the realm of embodied artificial intelligence, the reasoning capabilities of Large Language Models (LLMs) play a pivotal role. Although there are effective methods like program-of-thought prompting for LLMs which uses programming language to tackle complex reasoning tasks, the specific impact of code data on the improvement of reasoning capabilities remains under-explored. To address this gap, we propose complexity-impacted reasoning score (CIRS), which combines structural and logical attributes, to measure the correlation between code and reasoning abilities. Specifically, we use the abstract syntax tree to encode the structural information and calculate logical complexity by considering the difficulty and the cyclomatic complexity. Through an empirical analysis, we find not all code data of complexity can be learned or understood by LLMs. Optimal level of complexity is critical to the improvement of reasoning abilities by program-aided prompting. Then we design an auto-synthesizing and stratifying algorithm, and apply it to instruction generation for mathematical reasoning and code data filtering for code generation tasks. Extensive results demonstrates the effectiveness of our proposed approach. Code will be integrated into the EasyInstruct framework at https://github.com/zjunlp/EasyInstruct.

cs.CL↗

arXiv2023

Relphormer: Relational Graph Transformer for Knowledge Graph Representations

Transformers have achieved remarkable performance in widespread fields, including natural language processing, computer vision and graph mining. However, vanilla Transformer architectures have not yielded promising improvements in the Knowledge Graph (KG) representations, where the translational distance paradigm dominates this area. Note that vanilla Transformer architectures struggle to capture the intrinsically heterogeneous structural and semantic information of knowledge graphs. To this end, we propose a new variant of Transformer for knowledge graph representations dubbed Relphormer. Specifically, we introduce Triple2Seq which can dynamically sample contextualized sub-graph sequences as the input to alleviate the heterogeneity issue. We propose a novel structure-enhanced self-attention mechanism to encode the relational information and keep the semantic information within entities and relations. Moreover, we utilize masked knowledge modeling for general knowledge graph representation learning, which can be applied to various KG-based tasks including knowledge graph completion, question answering, and recommendation. Experimental results on six datasets show that Relphormer can obtain better performance compared with baselines. Code is available in https://github.com/zjunlp/Relphormer.

cs.CL↗

arXiv2023

Non-Fermi Liquids from Dipolar Symmetry Breaking

The emergence of fractonic topological phases and novel universality classes for quantum dynamics highlights the importance of dipolar symmetry in condensed matter systems. In this work, we study the properties of symmetry-breaking phases of the dipolar symmetries in fermionic models in various spatial dimensions. In such systems, fermions obtain energy dispersion through dipole condensation. Due to the nontrivial commutation between the translation symmetry and dipolar symmetry, the Goldstone modes of the dipolar condensate are strongly coupled to the dispersive fermions and naturally give rise to non-Fermi liquids at low energies. The IR description of the dipolar symmetry-breaking phase is analogous to the well-known theory of a Fermi surface coupled to an emergent U(1) gauge field. We also discuss the crossover behavior when the dipolar symmetry is slightly broken and the cases with anisotropic dipolar conservation.

cond-mat.str-el↗

arXiv2023

Fractonic Higher-Order Topological Phases in Open Quantum Systems

In this work, we study the generalization of decohered average symmetry-protected topological phases to open quantum systems with a combination of subsystem symmetries and global symmetries. In particular, we provide examples of two types of intrinsic average higher-order topological phases with average subsystem symmetries. A classification scheme for these phases based on generalized anomaly cancellation criteria of average symmetry is also discussed.

cond-mat.str-el↗

arXiv2023

Triggering Boundary Phase Transitions through Bulk Measurements in 2D Cluster States

We investigate the phase diagram at the boundary of an infinite two-dimensional cluster state subject to bulk measurements using tensor network methods. The state is subjected to uniform measurements $M = \cosθZ+\sinθX$ on the lower boundary qubits and in all bulk qubits. Our results show that the boundary of the system exhibits volume-law entanglement at the measurement angle $θ= π/2$ and area-law entanglement for any $θ< π/2$. Within the area-law phase, a phase transition occurs at $θ_c=1.371$. The phase with $θ\in(θ_c,π/2)$ is characterized by a noninjective matrix product state, which cannot be realized as the unique ground state of a one-dimensional local, gapped Hamiltonian. Instead, it resembles a cat state with spontaneous symmetry breaking. These findings demonstrate that the phase diagram of the boundary of a two-dimensional system can be more intricate than that of a standard one-dimensional system.

quant-ph↗

arXiv2023

$\mathbb Z_2$-Nontrivial Moiré Minibands and Interaction-Driven Quantum Anomalous Hall Insulators in Topological Insulator Based Moiré Heterostructures

We studied electronic band structure and topological property of a topological insulator thin film under a moiré superlattice potential to search for two-dimensional (2D) $\mathbb Z_2$ non-trivial isolated mini-bands. To model this system, we assume the Fermi energy inside the bulk band gap and thus consider an effective model Hamiltonian with only two surface states that are located at the top and bottom surfaces and strongly hybridized with each other. The moiré potential is generated by another layer of 2D insulating materials on top of topological insulator films. In this model, the lowest conduction (highest valence) mini-bands can be $\mathbb Z_2$ non-trivial when the minima (maxima) of the moiré potential approximately forms a hexagonal lattice with six-fold rotation symmetry. For the nontrivial conduction mini-band cases, the two lowest Kramers' pairs of conduction mini-bands both have nontrivial $\mathbb Z_2$ invariant in presence of inversion, while applying external gate voltages to break inversion leads to only the lowest Kramers' pair of mini-bands to be topologically non-trivial. The Coulomb interaction can drive the lowest conduction Kramers' mini-bands into the quantum anomalous Hall state when they are half-filled, which is further stabilized by breaking inversion symmetry. We propose the monolayer Sb$_{2}$ on top of Sb$_2$Te$_3$ thin films to realize our model based on results from the first principles calculations.

cond-mat.mes-hall↗

arXiv2023

Tele-Knowledge Pre-training for Fault Analysis

In this work, we share our experience on tele-knowledge pre-training for fault analysis, a crucial task in telecommunication applications that requires a wide range of knowledge normally found in both machine log data and product documents. To organize this knowledge from experts uniformly, we propose to create a Tele-KG (tele-knowledge graph). Using this valuable data, we further propose a tele-domain language pre-training model TeleBERT and its knowledge-enhanced version, a tele-knowledge re-training model KTeleBERT. which includes effective prompt hints, adaptive numerical data encoding, and two knowledge injection paradigms. Concretely, our proposal includes two stages: first, pre-training TeleBERT on 20 million tele-related corpora, and then re-training it on 1 million causal and machine-related corpora to obtain KTeleBERT. Our evaluation on multiple tasks related to fault analysis in tele-applications, including root-cause analysis, event association prediction, and fault chain tracing, shows that pre-training a language model with tele-domain data is beneficial for downstream tasks. Moreover, the KTeleBERT re-training further improves the performance of task models, highlighting the effectiveness of incorporating diverse tele-knowledge into the model.

cs.AI↗

arXiv2022

Multi-modal Protein Knowledge Graph Construction and Applications

Existing data-centric methods for protein science generally cannot sufficiently capture and leverage biology knowledge, which may be crucial for many protein tasks. To facilitate research in this field, we create ProteinKG65, a knowledge graph for protein science. Using gene ontology and Uniprot knowledge base as a basis, we transform and integrate various kinds of knowledge with aligned descriptions and protein sequences, respectively, to GO terms and protein entities. ProteinKG65 is mainly dedicated to providing a specialized protein knowledge graph, bringing the knowledge of Gene Ontology to protein function and structure prediction. We also illustrate the potential applications of ProteinKG65 with a prototype. Our dataset can be downloaded at https://w3id.org/proteinkg65.

q-bio.QM↗

arXiv2022

Classification and construction of interacting fractonic higher-order topological phases

The notion of higher-order topological phases can have interesting generalizations to systems with subsystem symmetries that exhibit fractonic dynamics for charged excitations. In this work, we systematically study the higher-order topological phases protected by a combination of subsystem symmetries and ordinary global symmetries in two and three-dimensional interacting boson systems, with some interacting fermionic examples.

cond-mat.str-el↗

arXiv2022

Decoding Measurement-Prepared Quantum Phases and Transitions: from Ising model to gauge theory, and beyond

Measurements allow efficient preparation of interesting quantum many-body states with long-range entanglement, conditioned on additional transformations based on measurement outcomes. Here, we demonstrate that the so-called conformal quantum critical points (CQCP) can be obtained by performing general single-site measurements in an appropriate basis on the cluster states in $d\geq2$. The equal-time correlators of the said states are described by correlation functions of certain $d$-dimensional classical models at finite temperatures and feature spatial conformal invariance. This establishes an exact correspondence between the measurement-prepared critical states and conformal field theories of a range of critical spin models, including familiar Ising models and gauge theories. Furthermore, by mapping the long-range entanglement structure of measured quantum states into the correlations of the corresponding thermal spin model, we rigorously establish the stability condition of the long-range entanglement in the measurement-prepared quantum states deviating from the ideal setting. Most importantly, we describe protocols to decode the resulting quantum phases and transitions without post-selection, thus transferring the exponential measurement complexity to a polynomial classical computation. Therefore, our findings suggest a novel mechanism in which a quantum critical wavefunction emerges, providing new practical ways to study quantum phases and conformal quantum critical points.

cond-mat.str-el↗

arXiv2022

OntoProtein: Protein Pretraining With Gene Ontology Embedding

Self-supervised protein language models have proved their effectiveness in learning the proteins representations. With the increasing computational power, current protein language models pre-trained with millions of diverse sequences can advance the parameter scale from million-level to billion-level and achieve remarkable improvement. However, those prevailing approaches rarely consider incorporating knowledge graphs (KGs), which can provide rich structured knowledge facts for better protein representations. We argue that informative biology knowledge in KGs can enhance protein representation with external knowledge. In this work, we propose OntoProtein, the first general framework that makes use of structure in GO (Gene Ontology) into protein pre-training models. We construct a novel large-scale knowledge graph that consists of GO and its related proteins, and gene annotation texts or protein sequences describe all nodes in the graph. We propose novel contrastive learning with knowledge-aware negative sampling to jointly optimize the knowledge graph and protein embedding during pre-training. Experimental results show that OntoProtein can surpass state-of-the-art methods with pre-trained protein language models in TAPE benchmark and yield better performance compared with baselines in protein-protein interaction and protein function prediction. Code and datasets are available in https://github.com/zjunlp/OntoProtein.

q-bio.BM↗

arXiv2022

Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners

Large-scale pre-trained language models have contributed significantly to natural language processing by demonstrating remarkable abilities as few-shot learners. However, their effectiveness depends mainly on scaling the model parameters and prompt design, hindering their implementation in most real-world applications. This study proposes a novel pluggable, extensible, and efficient approach named DifferentiAble pRompT (DART), which can convert small language models into better few-shot learners without any prompt engineering. The main principle behind this approach involves reformulating potential natural language processing tasks into the task of a pre-trained language model and differentially optimizing the prompt template as well as the target label with backpropagation. Furthermore, the proposed approach can be: (i) Plugged to any pre-trained language models; (ii) Extended to widespread classification tasks. A comprehensive evaluation of standard NLP tasks demonstrates that the proposed approach achieves a better few-shot performance. Code is available in https://github.com/zjunlp/DART.

cs.CL↗

arXiv2022

CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark

Artificial Intelligence (AI), along with the recent progress in biomedical language understanding, is gradually changing medical practice. With the development of biomedical language understanding benchmarks, AI applications are widely used in the medical field. However, most benchmarks are limited to English, which makes it challenging to replicate many of the successes in English for other languages. To facilitate research in this direction, we collect real-world biomedical data and present the first Chinese Biomedical Language Understanding Evaluation (CBLUE) benchmark: a collection of natural language understanding tasks including named entity recognition, information extraction, clinical diagnosis normalization, single-sentence/sentence-pair classification, and an associated online platform for model evaluation, comparison, and analysis. To establish evaluation on these tasks, we report empirical results with the current 11 pre-trained Chinese models, and experimental results show that state-of-the-art neural models perform by far worse than the human ceiling. Our benchmark is released at \url{https://tianchi.aliyun.com/dataset/dataDetail?dataId=95414&lang=en-us}.

cs.CL↗

arXiv2022

Interaction Enabled Fractonic Higher-Order Topological Phases

In this work, we present a collection of three-dimensional higher-order symmetry protected topological phases (HOSPTs) with gapless hinge modes that exist only in strongly interacting systems subject to subsystem symmetry constraints. We use a coupled wire construction to generate three families of microscopic lattice models: insulators with helical hinge modes, superconductors with chiral Majorana hinge modes, and fractionalized insulators with helical hinge modes that carry fractional charge. In particular, these HOSPTs do not require spatial symmetry protection, but are instead protected by subsystem symmetries, and support "fractonic" quasiparticle excitations that move within only a low-dimensional sub-manifold of the system. We analyze the anomaly structure for the boundary theory and the entanglement Hamiltonian, and show that the side surfaces of these HOSPTs, despite being partially gapped, exhibit symmetry anomalies, and can only be realized as the boundary of three-dimensional HOSPT phases.

cond-mat.str-el↗

arXiv2021

On Robustness and Bias Analysis of BERT-based Relation Extraction

Fine-tuning pre-trained models have achieved impressive performance on standard natural language processing benchmarks. However, the resultant model generalizability remains poorly understood. We do not know, for example, how excellent performance can lead to the perfection of generalization models. In this study, we analyze a fine-tuned BERT model from different perspectives using relation extraction. We also characterize the differences in generalization techniques according to our proposed improvements. From empirical experimentation, we find that BERT suffers a bottleneck in terms of robustness by way of randomizations, adversarial and counterfactual tests, and biases (i.e., selection and semantic). These findings highlight opportunities for future improvements. Our open-sourced testbed DiagnoseRE is available in \url{https://github.com/zjunlp/DiagnoseRE}.

cs.CL↗