arXiv subjects

Tao Xie

Publications and source records attributed to Tao Xie.

At least 91 records · Page 5Linked to original sources

arXiv2025

Visualization of intervalley coherent phase in PtSe2/HOPG heterojunction

Intervalley coherent (IVC) phase in graphene systems arises from the coherent superposition of wave functions of opposite valleys, whose direct microscopic visualization provides pivotal insight into the emergent physics but remains elusive. Here, we successfully visualize the IVC phase in a heterostructure of monolayer PtSe2 on highly oriented pyrolytic graphite. Using spectroscopic imaging scanning tunneling microscopy, we observe a Root3 by Root3 modulation pattern superimposed on the higher-order moire superlattice of the heterostructure, which correlates with a small gap opening around the Fermi level and displays an anti-phase real-space conductance distribution of the two gap edges. Such modulation pattern and small-gap vanish on the heterostructure of monolayer PtSe2 on bilayer-graphene-covered SiC substrate, due to the increased carrier density in the bilayer graphene. We provide a theoretical mechanism that the Root3 by Root3 modulation pattern originates from the IVC phase of few-layer graphene, which is magnified by the higher-order moire superlattice. Our work achieves visualization of the IVC phase, and develops an avenue for its generation and amplification via a moiré interface.

cond-mat.mes-hall↗

arXiv2024

Spin density wave in the bilayered nickelate La$_3$Ni$_2$O$_{7-δ}$ at ambient pressure

The recent discovery of high-temperature superconductivity in high-pressurized La$_3$Ni$_2$O$_{7-δ}$ has garnered significant attention. Using density functional theory, we investigate the magnetic properties of La$_3$Ni$_2$O$_{7-δ}$ at ambient pressure. Our calculations suggest that with $δ=0$, the double spin stripe phase is favored as the magnetic ground state. Oxygen vacancies may effectively turn nearest Ni spins into \textit{charge} sites. Consequently, with moderate $δ$ values, our theoretical magnetic ground state exhibits characteristics of both double spin stripe and spin-charge stripe configurations, providing a natural explanation to reconcile the seemingly contradictory experimental findings that suggest both the configurations as candidates for the spin-density-wave phase. With higher $δ$ values, we anticipate the ground state to become a spin-glass-like noncollinear magnetic phase with only short-range order. The oxygen vacancies are expected to significantly impact the magnetic excitations and the transition temperatures $T_{SDW}$. Notably, the magnetic ordering also induces concomitant charge ordering and orbital ordering, driven by spin-lattice coupling under the low symmetry magnetic order. We further offer a plausible explanation for the experimental observations that the measured $T_{SDW}$ appears insensitive to the variation of samples and the lack of direct evidence for long-range magnetic ordering.

cond-mat.mtrl-sci↗

arXiv2024

Effective Random Test Generation for Deep Learning Compilers

Deep learning compilers help address the difficulties of deploying deep learning models on diverse types of hardware. Testing deep learning compilers is highly crucial, because they are impacting countless AI applications that use them for model optimization and deployment. To test deep learning compilers, random testing, the testing method popularly used for compiler testing practices, faces the challenge of generating semantically valid test inputs, i.e., deep learning models that satisfy the semantic model specifications (in short as semantic specifications). To tackle this challenge, in this paper, we propose a novel approach named Isra, including a domain-specific constraint solver that resolves the constraints from the semantic specifications without backtracking. We implement and apply our approach to three popular real-world deep learning compilers including TVM, Glow, and a commercial compiler named SophGo. The evaluation results show that Isra is more effective than the state-of-the-art approaches and the baseline approaches on constructing valid test inputs for compiler-bug detection, and Isra successfully finds 24 previously unknown bugs in released versions of the three compilers. These results indicate Isra's effectiveness and practical value.

cs.SE↗

arXiv2024

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool

Large language model (LLM) serving has transformed from stateless to stateful systems, utilizing techniques like context caching and disaggregated inference. These optimizations extend the lifespan and domain of the KV cache, necessitating a new architectural approach. We present MemServe, a unified system that integrates both inter-request and intra-request optimizations. MemServe introduces MemPool, an elastic memory pool managing distributed memory and KV caches across serving instances. Using MemPool APIs, MemServe combines context caching with disaggregated inference for the first time, supported by a global scheduler that enhances cache reuse through a global prompt tree-based locality-aware policy. Tests show that MemServe significantly improves job completion time and time-to-first-time.

cs.DC↗

arXiv2024

Validity-Preserving Delta Debugging via Generator Trace Reduction

Reducing test inputs that trigger bugs is crucial for efficient debugging. Delta debugging is the most popular approach for this purpose. When test inputs need to conform to certain specifications, existing delta debugging practice encounters a validity problem: it blindly applies reduction rules, producing a large number of invalid test inputs that do not satisfy the required specifications. This overall diminishing effectiveness and efficiency becomes even more pronounced when the specifications extend beyond syntactical structures. Our key insight is that we should leverage input generators, which are aware of these specifications, to generate valid reduced inputs, rather than straightforwardly performing reduction on test inputs. In this paper, we propose a generator-based delta debugging method, namely GReduce, which derives validity-preserving reducers. Specifically, given a generator and its execution, demonstrating how the bug-inducing test input is generated, GReduce searches for other executions on the generator that yield reduced, valid test inputs. The evaluation results on five benchmarks (i.e., graphs, DL models, JavaScript programs, SymPy, and algebraic data types) show that GReduce substantially outperforms state-of-the-art syntax-based reducers including Perses and T-PDD, and also outperforms QuickCheck, SmartCheck, as well as the state-of-the-art choice-sequence-based reducer Hypothesis, demonstrating the effectiveness, efficiency, and versatility of GReduce.

cs.SE↗

arXiv2024

InfiBench: Evaluating the Question-Answering Capabilities of Code Large Language Models

Large Language Models for code (code LLMs) have witnessed tremendous progress in recent years. With the rapid development of code LLMs, many popular evaluation benchmarks, such as HumanEval, DS-1000, and MBPP, have emerged to measure the performance of code LLMs with a particular focus on code generation tasks. However, they are insufficient to cover the full range of expected capabilities of code LLMs, which span beyond code generation to answering diverse coding-related questions. To fill this gap, we propose InfiBench, the first large-scale freeform question-answering (QA) benchmark for code to our knowledge, comprising 234 carefully selected high-quality Stack Overflow questions that span across 15 programming languages. InfiBench uses four types of model-free automatic metrics to evaluate response correctness where domain experts carefully concretize the criterion for each question. We conduct a systematic evaluation for over 100 latest code LLMs on InfiBench, leading to a series of novel and insightful findings. Our detailed analyses showcase potential directions for further advancement of code LLMs. InfiBench is fully open source at https://infi-coder.github.io/infibench and continuously expanding to foster more scientific and systematic practices for code LLM evaluation.

cs.SE↗

arXiv2024

Stripe magnetic order and field-induced quantum criticality in the perfect triangular-lattice antiferromagnet CsCeSe$_2$

The two-dimensional triangular-lattice antiferromagnet (TLAF) is a textbook example of frustrated magnetic systems. Despite its simplicity, the TLAF model exhibits a highly rich and complex magnetic phase diagram, featuring numerous distinct ground states that can be stabilized through frustrated next-nearest-neighbor couplings or anisotropy. In this paper, we report low-temperature magnetic properties of the TLAF material CsCeSe$_2$. The inelastic neutron scattering (INS) together with specific heat measurements and density functional theory calculations of crystalline electric field suggest that the ground state of Ce ions is a Kramers doublet with strong easy-plane anisotropy. Elastic neutron scattering measurements demonstrate the presence of stripe-$yz$ magnetic order that develops below $T_{\rm N} = 0.35$ K, with the zero-field ordered moment of $m_{\rm Ce} \approx 0.65~μ_{\rm B}$. Application of magnetic field first increases the ordering temperature by about 20% at the intermediate field region and eventually suppresses the stripe order in favor of the field-polarized ferromagnetic state via a continuous quantum phase transition (QPT). The field-induced response demonstrates sizable anisotropy for different in-plane directions, $\mathbf{B}\parallel{}\mathbf{a}$ and $\mathbf{B}\perp{}\mathbf{a}$, which indicates the presence of bond-dependent coupling in the spin Hamiltonian. We further show theoretically that the presence of anisotropic bond-dependent interactions can change the universality class of QPT for $\mathbf{B}\parallel{}\mathbf{a}$ and $\mathbf{B}\perp{}\mathbf{a}$.

cond-mat.str-el↗

arXiv2024

Quantum Spin Dynamics Due to Strong Kitaev Interactions in the Triangular-Lattice Antiferromagnet CsCeSe$_2$

The extraordinary properties of the Kitaev model have motivated an intense search for new physics in materials that combine geometrical and bond frustration. In this work, we employ inelastic neutron scattering, spin wave theory, and exact diagonalization to study the spin dynamics in the perfect triangular-lattice antiferromagnet (TLAF) CsCeSe$_2$. This material orders into a stripe phase, which is demonstrated to arise as a consequence of the off-diagonal bond-dependent terms in the spin Hamiltonian. By studying the spin dynamics at intermediate fields, we identify an interaction between the single-magnon state and the two-magnon continuum that causes decay of coherent magnon excitations, level repulsion, and transfer of spectral weight to the continuum that are controlled by the strength of the magnetic field. Our results provide a microscopic mechanism for the stabilization of the stripe phase in TLAF and show how complex many-body physics can be present in the spin dynamics in a magnet with strong Kitaev coupling even in an ordered ground state.

cond-mat.str-el↗

arXiv2024

PSE-Net: Channel Pruning for Convolutional Neural Networks with Parallel-subnets Estimator

Channel Pruning is one of the most widespread techniques used to compress deep neural networks while maintaining their performances. Currently, a typical pruning algorithm leverages neural architecture search to directly find networks with a configurable width, the key step of which is to identify representative subnet for various pruning ratios by training a supernet. However, current methods mainly follow a serial training strategy to optimize supernet, which is very time-consuming. In this work, we introduce PSE-Net, a novel parallel-subnets estimator for efficient channel pruning. Specifically, we propose a parallel-subnets training algorithm that simulate the forward-backward pass of multiple subnets by droping extraneous features on batch dimension, thus various subnets could be trained in one round. Our proposed algorithm facilitates the efficiency of supernet training and equips the network with the ability to interpolate the accuracy of unsampled subnets, enabling PSE-Net to effectively evaluate and rank the subnets. Over the trained supernet, we develop a prior-distributed-based sampling algorithm to boost the performance of classical evolutionary search. Such algorithm utilizes the prior information of supernet training phase to assist in the search of optimal subnets while tackling the challenge of discovering samples that satisfy resource constraints due to the long-tail distribution of network configuration. Extensive experiments demonstrate PSE-Net outperforms previous state-of-the-art channel pruning methods on the ImageNet dataset while retaining superior supernet training efficiency. For example, under 300M FLOPs constraint, our pruned MobileNetV2 achieves 75.2% Top-1 accuracy on ImageNet dataset, exceeding the original MobileNetV2 by 2.6 units while only cost 30%/16% times than BCNet/AutoAlim.

cs.CV↗

arXiv2024

SWE-bench-java: A GitHub Issue Resolving Benchmark for Java

GitHub issue resolving is a critical task in software engineering, recently gaining significant attention in both industry and academia. Within this task, SWE-bench has been released to evaluate issue resolving capabilities of large language models (LLMs), but has so far only focused on Python version. However, supporting more programming languages is also important, as there is a strong demand in industry. As a first step toward multilingual support, we have developed a Java version of SWE-bench, called SWE-bench-java. We have publicly released the dataset, along with the corresponding Docker-based evaluation environment and leaderboard, which will be continuously maintained and updated in the coming months. To verify the reliability of SWE-bench-java, we implement a classic method SWE-agent and test several powerful LLMs on it. As is well known, developing a high-quality multi-lingual benchmark is time-consuming and labor-intensive, so we welcome contributions through pull requests or collaboration to accelerate its iteration and refinement, paving the way for fully automated programming.

cs.SE↗

arXiv2024

Regulated magnetic anisotropy and charge density wave in uniformly fabricated Janus CrTeSe monolayer

Two-dimensional materials with Janus structure host novel physical properties due to their inversional symmetry breaking. However, it remains elusive to synthesize Janus monolayer crystals with tailored long-range magnetic orders. Here, we have developed a general method to fabricate uniform Janus CrTeSe monolayers by selective selenization of preformed CrTe2 monolayers with molecular beam epitaxy. The uniform Janus structure of CrTeSe with high crystal quality is confirmed by high-resolution scanning transmission electron microscopy. Spin-polarized scanning tunneling microscopy/spectroscopy measurements unveil that the Janus CrTeSe undergoes a charge density wave (CDW) transition and a robust antiferromagnetic order. The magnetic anisotropy of CrTeSe is drastically altered compared to monolayer CrTe2 by the breaking symmetries induced from the Janus structure and the CDW transition, as is substantiated with first principles calculations. Our research achieves the construction of large-area Janus structures, and artificially tailors the electronic and magnetic properties of Janus systems at the two-dimensional limit.

cond-mat.mtrl-sci↗

arXiv2024

L^2CL: Embarrassingly Simple Layer-to-Layer Contrastive Learning for Graph Collaborative Filtering

Graph neural networks (GNNs) have recently emerged as an effective approach to model neighborhood signals in collaborative filtering. Towards this research line, graph contrastive learning (GCL) demonstrates robust capabilities to address the supervision label shortage issue through generating massive self-supervised signals. Despite its effectiveness, GCL for recommendation suffers seriously from two main challenges: i) GCL relies on graph augmentation to generate semantically different views for contrasting, which could potentially disrupt key information and introduce unwanted noise; ii) current works for GCL primarily focus on contrasting representations using sophisticated networks architecture (usually deep) to capture high-order interactions, which leads to increased computational complexity and suboptimal training efficiency. To this end, we propose L2CL, a principled Layer-to-Layer Contrastive Learning framework that contrasts representations from different layers. By aligning the semantic similarities between different layers, L2CL enables the learning of complex structural relationships and gets rid of the noise perturbation in stochastic data augmentation. Surprisingly, we find that L2CL, using only one-hop contrastive learning paradigm, is able to capture intrinsic semantic structures and improve the quality of node representation, leading to a simple yet effective architecture. We also provide theoretical guarantees for L2CL in minimizing task-irrelevant information. Extensive experiments on five real-world datasets demonstrate the superiority of our model over various state-of-the-art collaborative filtering methods. Our code is available at https://github.com/downeykking/L2CL.

cs.IR↗

arXiv2024

QVD: Post-training Quantization for Video Diffusion Models

Recently, video diffusion models (VDMs) have garnered significant attention due to their notable advancements in generating coherent and realistic video content. However, processing multiple frame features concurrently, coupled with the considerable model size, results in high latency and extensive memory consumption, hindering their broader application. Post-training quantization (PTQ) is an effective technique to reduce memory footprint and improve computational efficiency. Unlike image diffusion, we observe that the temporal features, which are integrated into all frame features, exhibit pronounced skewness. Furthermore, we investigate significant inter-channel disparities and asymmetries in the activation of video diffusion models, resulting in low coverage of quantization levels by individual channels and increasing the challenge of quantization. To address these issues, we introduce the first PTQ strategy tailored for video diffusion models, dubbed QVD. Specifically, we propose the High Temporal Discriminability Quantization (HTDQ) method, designed for temporal features, which retains the high discriminability of quantized features, providing precise temporal guidance for all video frames. In addition, we present the Scattered Channel Range Integration (SCRI) method which aims to improve the coverage of quantization levels across individual channels. Experimental validations across various models, datasets, and bit-width settings demonstrate the effectiveness of our QVD in terms of diverse metrics. In particular, we achieve near-lossless performance degradation on W8A8, outperforming the current methods by 205.12 in FVD.

cs.CV↗

arXiv2024

Helical Spin Dynamics in Commensurate Magnets: a Study on Brochantite, Cu$_4$SO$_4$(OH)$_6$

We report the direct observation of a commensurate-ordered antiferromagnetic (AFM) state but incommensurate helical spin dynamics in the natural mineral brochantite Cu$_4$SO$_4$(OH)$_6$ through neutron diffraction and neutron spectroscopy measurements. Inelastic neutron scattering measurements reveal magnon-like excitations with considerable dispersion along the c-axis and almost flat branches in other principal directions, indicating the strong one-dimensional character of the magnetic correlations. We experimentally observe the effect of the uniform Dzyaloshinskii-Moriya (DM) interaction, which elevates the degeneracy of the spin-wave modes shifting them in opposite directions in reciprocal space. The system has a commensurate AFM ground state, stabilized by the anisotropic symmetric Heisenberg exchange interactions, and quasi-one-dimensional chiral spin dynamics due to the antisymmetric DM interaction. Employing linear spin-wave theory, we were able to construct an effective Heisenberg Hamiltonian. We quantify both the symmetric exchange parameters and the DM vector components in Cu$_4$SO$_4$(OH)$_6$ and determine the mechanism of the magnetic frustration. Our work provides detailed insights into the complex dynamics of the spin chain in the presence of uniform DM interaction.

cond-mat.str-el↗

arXiv2024

High-Resolution Spectroscopy of the Intermediate Impurity States near a Quantum Phase Transition

The intermediate behavior near a quantum phase transition is crucial for understanding the quantum criticality of various competing phases and their separate origins, yet remains unexplored for the multiple Yu-Shiba-Rusinov (YSR) states. Here, we investigated the detailed spectroscopic change of the exchange coupling-dependent YSR states near a quantum phase transition. The initially developed one pair of YSR states, induced by the Fe vacancy in monolayer Fe(Te,Se) superconductor, are clearly resolved with high resolution showing an evolution into two pairs of YSR peaks yet with dichotomy in their spectral features. Interestingly, while the lower-energy YSR branch enters the quantum phase transition region, the higher-lying one remains rigidly away from the lower-energy counterpart with a constant energy difference. Spectral weight analysis of the higher-energy branch yields an exponential dependence on the exchange coupling, which can be well rationalized by taking the two pairs of YSR states as a result of field splitting by the magnetic anisotropy. Our results unveil the intermediate region of a quantum phase transition with a magnetic anisotropy-induced splitting of the YSR resonance, and highlight a prospect for developing functional electronics based on the flexibly controllable multiple quantum states.

cond-mat.supr-con↗

arXiv2024

Foundation Model Engineering: Engineering Foundation Models Just as Engineering Software

By treating data and models as the source code, Foundation Models (FMs) become a new type of software. Mirroring the concept of software crisis, the increasing complexity of FMs making FM crisis a tangible concern in the coming decade, appealing for new theories and methodologies from the field of software engineering. In this paper, we outline our vision of introducing Foundation Model (FM) engineering, a strategic response to the anticipated FM crisis with principled engineering methodologies. FM engineering aims to mitigate potential issues in FM development and application through the introduction of declarative, automated, and unified programming interfaces for both data and model management, reducing the complexities involved in working with FMs by providing a more structured and intuitive process for developers. Through the establishment of FM engineering, we aim to provide a robust, automated, and extensible framework that addresses the imminent challenges, and discovering new research opportunities for the software engineering field.

cs.SE↗

arXiv2024

Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCache

Large Language Models (LLMs) demonstrate substantial potential across a diverse array of domains via request serving. However, as trends continue to push for expanding context sizes, the autoregressive nature of LLMs results in highly dynamic behavior of the attention layers, showcasing significant differences in computational characteristics and memory requirements from the non-attention layers. This presents substantial challenges for resource management and performance optimization in service systems. Existing static model parallelism and resource allocation strategies fall short when dealing with this dynamicity. To address the issue, we propose Infinite-LLM, a novel LLM serving system designed to effectively handle dynamic context lengths. Infinite-LLM disaggregates attention layers from an LLM's inference process, facilitating flexible and independent resource scheduling that optimizes computational performance and enhances memory utilization jointly. By leveraging a pooled GPU memory strategy across a cluster, Infinite-LLM not only significantly boosts system throughput but also supports extensive context lengths. Evaluated on a dataset with context lengths ranging from a few to 2000K tokens across a cluster with 32 A100 GPUs, Infinite-LLM demonstrates throughput improvement of 1.35-3.4x compared to state-of-the-art methods, enabling efficient and elastic LLM deployment.

cs.DC↗

arXiv2024

Decomposition for Enhancing Attention: Improving LLM-based Text-to-SQL through Workflow Paradigm

In-context learning of large-language models (LLMs) has achieved remarkable success in the field of natural language processing, while extensive case studies reveal that the single-step chain-of-thought prompting approach faces challenges such as attention diffusion and inadequate performance in complex tasks like text-to-SQL. To improve the contextual learning capabilities of LLMs in text-to-SQL, a workflow paradigm method is proposed, aiming to enhance the attention and problem-solving scope of LLMs through decomposition. Specifically, the information determination module for eliminating redundant information and the brand-new prompt structure based on problem classification greatly enhance the model's attention. Additionally, the inclusion of self-correction and active learning modules greatly expands the problem-solving scope of LLMs, hence improving the upper limit of LLM-based approaches. Extensive experiments conducted on three datasets demonstrate that our approach outperforms other methods by a significant margin. About 2-3 percentage point improvements compared to the existing baseline on the Spider Dev, Spider-Realistic, and Bird Dev datasets and new SOTA results on the Spider Test dataset are achieved. Our code is available on GitHub: \url{https://github.com/FlyingFeather/DEA-SQL}.

cs.CL↗