Searcharxiv⌕ Search

arXiv subjects

Chao Yang

Publications and source records attributed to Chao Yang.

At least 109 records · Page 6Linked to original sources

Privacy Preservation by Local Design in Cooperative Networked Control Systems

In this paper, we study the privacy preservation problem in a cooperative networked control system, which has closed-loop dynamics, working for the task of linear quadratic Guassian (LQG) control. The system consists of a user and a server: the user owns the plant to control, while the server provides computation capability, and the user employs the server to compute control inputs for it. To enable the server's computation, the user needs to provide the measurements of the plant states to the server, who then calculates estimates of the states, based on which the control inputs are computed. However, the user regards the states as privacy, and makes an interesting request: the user wants the server to have "incorrect" knowledge of the state estimates rather than the true values. Regarding that, we propose a novel design methodology for the privacy preservation, in which the privacy scheme is locally equipped at the user side not open to the server, which manages to create a deviation in the server's knowledge of the state estimates from the true values. However, this methodology also raises significant challenges: in a closed-loop dynamic system, when the server's seized knowledge is incorrect, the system's behavior becomes complex to analyze; even the stability of the system becomes questionable, as the incorrectness will accumulate through the closed loop as time evolves. In this paper, we succeed in showing that the performance loss in LQG control caused by the proposed privacy scheme is bounded by rigorous mathematical proofs, which convinces the availability of the proposed design methodology. We also propose an associated novel privacy metric and obtain the analytical result on evaluating the privacy performance. Finally, we study the performance trade-off between privacy and control, where the accordingly proposed optimization problems are solved by numerical methods efficiently.

eess.SY↗

Numerically Exact Configuration Interaction at Quadrillion-Determinant Scale

The combinatorial scaling of configuration interaction (CI) has long restricted its applicability to only the simplest molecular systems. Here, we report the first numerically exact CI calculation exceeding one quadrillion ($10^{15}$) determinants, enabled by categorical compression within the small-tensor-product distributed active space (STP-DAS) framework. As a demonstration, we converged the relativistic complete active space CI (CASCI) ground state of HBrTe involving over $10^{15}$ complex-valued 2-spinor determinants in under 34.5 hours (time-to-completion) using 1000 nodes, representing the largest CASCI calculation reported to date. Additionally, we achieved $\boldsymbolσ$-build times of just 5 minutes for systems with approximately 150 billion complex-valued 2-spinor determinants using only a few compute nodes. Extensive benchmarks confirm that the method retains numerical exactness with drastically reduced resource demands. Compared to previous state-of-the-art CI calculations, this work represents a 3-orders-of-magnitude increase in CI space, a 6-orders-of-magnitude increase in FLOP count, and a 6-orders-of-magnitude improvement in computational speed. By introducing a numerically exact, categorically compressed representation of the CI expansion vectors and reformulating the $\boldsymbolσ$-build accordingly, we eliminate memory bottlenecks associated with storing excitation lists and CI vectors while significantly reducing computational cost. A compression-compatible preconditioner further enhances performance by generating compressed CI expansion vectors throughout Davidson iterations. This work establishes a new computational frontier for numerically exact CI methods, enabling chemically and physically accurate simulations of strongly correlated, spin-orbit coupled systems previously thought to be beyond reach.

physics.chem-ph↗

Computing excited states with isometric tensor networks in two-dimensions

We present a new subspace iteration method for computing low-lying eigenpairs (excited states) of high-dimensional quantum many-body Hamiltonians with nearest neighbor interactions on two-dimensional lattices. The method is based on a new block isometric projected entangled pair state (block-isoPEPS) ansatz that generalizes the block matrix product state (MPS) framework, widely used for Hamiltonians defined on one-dimensional chains, to two-dimensions. The proposed block-isoPEPS ansatz offers several attractive features for PEPS-based algorithms, including exact block orthogonalization, controlled local truncation via singular value decompositions, and efficient evaluation of observables. We demonstrate the proposed inexact subspace iteration for block-isoPEPS by computing excitations of the two-dimensional transverse-field Ising and Heisenberg models and compare our results with existing PEPS methods. Our results demonstrate that block isometric tensor networks provide a scalable framework for studying excitations in quantum many-body systems beyond one dimension.

math.NA↗

HaTT: Hadamard avoiding TT recompression

The Hadamard product of tensor train (TT) tensors is a fundamental nonlinear operation in scientific computing and data analysis. However, due to its tendency to significantly increase TT ranks, the Hadamard product poses a major computational challenge in TT tensor-based algorithms. To address this, it is crucial to develop recompression algorithms that mitigate the effects of this rank increase. Existing recompression algorithms require an explicit representation of the Hadamard product, resulting in high computational and storage costs. In this work, we propose a Hadamard avoiding TT recompression (HaTT) algorithm, which reduces both computational complexity and storage requirements. By leveraging the structure of the Hadamard product in TT tensors and exploiting its Hadamard product-free property, the HaTT algorithm achieves significantly lower complexity compared to existing TT recompression methods. This is confirmed through both complexity analysis and numerical experiments. Furthermore, the HaTT algorithm is applied to solve the Allen--Cahn equation, achieving substantial speedup over existing TT recompression algorithms without sacrificing accuracy.

math.NA↗

Inexact subspace projection methods for low-rank tensor eigenvalue problems

We propose inexact subspace iteration for solving high-dimensional eigenvalue problems with low-rank structure. Inexactness stems from low-rank compression, enabling efficient representation of high-dimensional vectors in a low-rank tensor format. A primary challenge in these methods is that standard operations, such as matrix-vector products and linear combinations, increase tensor rank, necessitating rank truncation and hence approximation. We compare the proposed methods with an existing inexact Lanczos method with low-rank compression. This method constructs an approximate orthonormal Krylov basis, which is often difficult to represent accurately in low-rank tensor formats, even when the eigenvectors themselves exhibit low-rank structure. In contrast, inexact subspace iteration uses approximate eigenvectors (Ritz vectors) directly as a subspace basis, bypassing the need for an orthonormal Krylov basis. Our analysis and numerical experiments demonstrate that inexact subspace iteration is much more robust to rank-truncation errors compared to the inexact Lanczos method. We also demonstrate that rank-truncated subspace iteration can converge for problems where the DMRG method stagnates. Furthermore, the proposed subspace iteration methods do not require a Hermitian matrix, in contrast to Lanczos and DMRG, which are designed specifically for Hermitian matrices.

math.NA↗

Dissipation-Driven Transition of Particles from Dispersive to Flat Bands

Flat bands (FBs) play a crucial role in condensed matter physics, offering an ideal platform to study strong correlation effects and enabling applications in diffraction-free photonics and quantum devices. However, the study and application of FB properties are susceptible to interference from dispersive bands. Here, we explore the impact of bond dissipation on systems hosting both flat and dispersive bands by calculating the steady-state density matrix. We demonstrate that bond dissipation can drive particles from dispersive bands into FBs and establish the general conditions for this phenomenon to occur. Our results demonstrate that dissipation can facilitate FB preparation, property measurement, and utilization. This opens a new avenue for exploring FB physics in open quantum systems, with potential implications for strongly correlated physics.

quant-ph↗

A Flexible Programmable Pipeline Parallelism Framework for Efficient DNN Training

Pipeline parallelism is an essential distributed parallelism method. Increasingly complex and diverse DNN models necessitate meticulously customized pipeline schedules for performance. However, existing practices typically rely on predefined schedules, each with strengths, but fail to adapt automatically to the emerging model architectures. Exploring novel high-efficiency schedules is daunting due to the enormous and varying schedule space. Besides, manually implementing schedules can be challenging due to the onerous coding burdens and constantly changing needs. Unfortunately, existing frameworks have limitations in automated schedule exploration and lack flexibility and controllability. This paper presents FlexPipe, a programmable pipeline parallelism framework with enhanced productivity, programmability, debuggability, and ease of tuning. FlexPipe has two main components: a succinct domain-specific language (DSL) and an automated scheduler. FlexPipe enables automated schedule exploration for various parallel scenarios within a broad spectrum of schedule types at a small search cost. Besides, users can swiftly develop and customize schedules using the FlexPipe DSL, which embodies flexible controllability in the pipeline order of micro-batch computations over stages. It also provides convenient mechanisms to include new operations in schedules to meet changing demands. Our evaluation results demonstrate that FlexPipe achieves up to 2.28X performance speedup compared to the popular large-scale parallel framework Megtron-LM, and gains up to 1.49X performance speedup compared to the state-of-the-art automated pipeline parallelism framework.

cs.DC↗

Zeppelin: Balancing Variable-length Workloads in Data Parallel Large Model Training

Training large language models (LLMs) with increasingly long and varying sequence lengths introduces severe load imbalance challenges in large-scale data-parallel training. Recent frameworks attempt to mitigate these issues through data reorganization or hybrid parallel strategies. However, they often overlook how computational and communication costs scale with sequence length, resulting in suboptimal performance. We identify three critical challenges: (1) varying computation-to-communication ratios across sequences of different lengths in distributed attention, (2) mismatch between static NIC-GPU affinity and dynamic parallel workloads, and (3) distinct optimal partitioning strategies required for quadratic attention versus linear components. To address these challenges, we present Zeppelin, a novel training system that integrates three key techniques: (1) a hierarchical sequence partitioning method for the attention module that reduces communication overhead and balances computation, supported by an efficient attention engine that applies divergent parallel strategies; (2) a routing layer that orchestrates inter-node transfers to fully utilize NIC bandwidth; and (3) a remapping layer that transforms sequence layouts between attention and linear modules, ensuring high computational efficiency across both. Comprehensive evaluations across diverse configurations show that Zeppelin delivers an average 2.80x speedup over state-of-the-art methods.

cs.DC↗

Y(z)-injective vertex superalgebras and Hopf actions

This paper investigates Y(z)-injective vertex superalgebras. We first establish that two fundamental classes of vertex superalgebras -- simple ones and those admitting a PBW basis -- are Y(z)-injective. We then study actions of Hopf algebras on Y(z)-injective vertex superalgebras and prove that every finite-dimensional Hopf algebra acting inner faithfully on such algebras must be a group algebra. As a direct consequence, the study of the structure and representation theory of fixed-point subalgebras under finite-dimensional Hopf algebra actions reduces to that under group actions.

math.QA↗

SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention

Large language models (LLMs) now support extremely long context windows, but the quadratic complexity of vanilla attention results in significantly long Time-to-First-Token (TTFT) latency. Existing approaches to address this complexity require additional pretraining or finetuning, and often sacrifice model accuracy. In this paper, we first provide both theoretical and empirical foundations for near-lossless sparse attention. We find dynamically capturing head-specific sparse patterns at runtime with low overhead is crucial. To address this, we propose SampleAttention, an adaptive structured and near-lossless sparse attention. Leveraging observed significant sparse patterns, SampleAttention attends to a fixed percentage of adjacent tokens to capture local window patterns, and employs a two-stage query-guided key-value filtering approach, which adaptively select a minimum set of key-values with low overhead, to capture column stripe patterns. Comprehensive evaluations show that SampleAttention can seamlessly replace vanilla attention in off-the-shelf LLMs with nearly no accuracy loss, and reduces TTFT by up to $2.42\times$ compared with FlashAttention.

cs.CL↗

QuGStep: Refining Step Size Selection in Gradient Estimation for Variational Quantum Algorithms

Variational quantum algorithms (VQAs) offer a promising approach to solving computationally demanding problems by combining parameterized quantum circuits with classical optimization. Estimating probabilistic outcomes on quantum hardware requires repeated measurements (shots). However, in practice, the limited shot budget introduces significant noise in the evaluation of the objective function. Gradient estimation in VQAs often relies on the finite-difference, which evaluates the noisy objective function at perturbed circuit parameter values. The accuracy of this estimation is highly dependent on the choice of step size for these perturbations. An inappropriate step size can exacerbate the impact of noise, causing inaccurate gradient estimates and hindering the classical optimization in VQAs. This paper proposes QuGStep, an algorithm that addresses the challenge of determining the appropriate step size for finite-difference gradient estimation under a shot budget. QuGStep is grounded in a theorem that proves the optimal step size, which accounts for the shot budget, minimizes the error bound in gradient estimation using finite differences. Numerical experiments approximating the ground state energy of several molecules demonstrate that QuGStep can identify the appropriate step size for the given shot budget to obtain effective gradient estimation. Notably, the step size identified by QuGStep achieved convergence to the ground state energy with over 94% fewer shots compared to using a default step size (i.e., step size of 0.01). These findings highlight the potential of QuGStep to improve the practical deployment and scalability of quantum computing technologies.

quant-ph↗

Numerical Optimization for Tensor Disentanglement

Tensor networks provide compact and scalable representations of high-dimensional data, enabling efficient computation in fields such as quantum physics, numerical partial differential equations (PDEs), and machine learning. This paper focuses on tensor disentangling, the task of identifying transformations that reduce bond dimensions by exploiting gauge freedom in the network. We formulate this task as an optimization problem over orthogonal matrices acting on a single tensor's indices, aiming to minimize the rank of its matricized form. We present Riemannian optimization methods and a joint optimization framework that alternates between optimizing the orthogonal transformation for a fixed low-rank approximation and optimizing the low-rank approximation for a fixed orthogonal transformation, offering a competitive alternative when the target rank is known. To seek the often unknown optimal rank, we introduce a binary search strategy integrated with the disentangling procedure. Numerical experiments on random tensors and tensors in an approximate isometric tensor network state are performed to compare different optimization methods and explore the possibility of combining different methods in a hybrid approach.

math.NA↗

PeTTO: Leveraging GPUs to Accelerate Topology Optimization with the Pseudo-Transient Methods

We present a Pseudo-Transient Topology Optimization (PeTTO) approach that can leverage graphics processing units (GPUs) to efficiently solve single-material and multi-material topology optimization problems. By integrating PeTTO with phase field methods, the partial differential equations (PDEs) constrained optimization problem in topology optimization is transformed into a set of time dependent PDEs, which can be analyzed using the knowledge of transient physics. The sensitivities with respect to the design variable are calculated with the automatic differentiation which help avoid tedious and error-prone manual derivations. The overall system of equations is efficiently solved using a hybrid of the pseudo-transient method and the accelerated pseudo-transient method, balancing the convergence rate and numerical stability. A variety of numerical examples are presented to demonstrate the effectiveness and efficiency of the proposed PeTTO approach. These examples cover different physics scenarios including mechanical and thermal problems, as well as single-material and multi-materials cases in both 2D and 3D. The numerical results show a 40- to 50-fold speedup when running the same PeTTO code on a single GPU compared to desktop CPUs. This work helps bridge the gap between high-performance computing and topology optimization, potentially enabling faster and better designs for real-world problems.

math.NA↗

Dissipation-induced Half Quantized Conductance in One-dimensional Topological Systems

Quantized conductance from topologically protected edge states is a hallmark of two-dimensional topological phases. In contrast, edge states in one-dimensional (1D) topological systems cannot transmit current across the insulating bulk, rendering their topological nature invisible in transport. In this work, we investigate the transport properties of the Su-Schrieffer-Heeger model with gain and loss, and show that the zero-energy conductance exhibits qualitatively distinct behaviors between the topologically trivial and nontrivial phases, depending on the hybridization and dissipation strengths. Crucially, we analytically demonstrate that the conductance can become half-quantized in the topologically nontrivial phase, a feature absent in the trivial phase. We further show that the half quantization predominantly originates from transport channels involving gain/loss and edge states. Our results uncover a new mechanism for realizing quantized transport in 1D topological systems and highlight the nontrivial role of dissipation in enabling topological signatures in open quantum systems.

cond-mat.mes-hall↗

Emergent Response Planning in LLMs

In this work, we argue that large language models (LLMs), though trained to predict only the next token, exhibit emergent planning behaviors: $\textbf{their hidden representations encode future outputs beyond the next token}$. Through simple probing, we demonstrate that LLM prompt representations encode global attributes of their entire responses, including $\textit{structure attributes}$ (e.g., response length, reasoning steps), $\textit{content attributes}$ (e.g., character choices in storywriting, multiple-choice answers at the end of response), and $\textit{behavior attributes}$ (e.g., answer confidence, factual consistency). In addition to identifying response planning, we explore how it scales with model size across tasks and how it evolves during generation. The findings that LLMs plan ahead for the future in their hidden representations suggest potential applications for improving transparency and generation control.

cs.CL↗

On the Undecidability of Tiling the $3$-dimensional Space with a Set of $3$ Polycubes

Translational tiling problems are among the most fundamental and representative undecidable problems in all fields of mathematics. Greenfeld and Tao obtained two remarkable results on the undecidability of translational tiling in recent years. One is the existence of an aperiodic monotile in a space of sufficiently large dimension. The other is the undecidability of translational tiling of periodic subsets of space with a single tile, provided that the dimension of the space is part of the input. These two results support the following conjecture: there is a fixed dimension $n$ such that translational tiling with a single tile is undecidable. One strategy towards solving this conjecture is to prove the undecidability of translational tiling of a fixed dimension space with a set of $k$ tiles, for a positive integer $k$ as small as possible. In this paper, it is shown that translational tiling the $3$-dimensional space with a set of $3$ polycubes is undecidable.

math.CO↗

DeltaDiff: Reality-Driven Diffusion with AnchorResiduals for Faithful SR

Recently, the transfer application of diffusion models in super-resolu-tion tasks has faced the problem ofdecreased fidelity. Due to the inherent randomsampling characteristics ofdiffusion models, direct application in super-resolu-tion tasks can result in generated details deviating from the true distribution ofhigh-resolution images. To address this, we propose DeltaDiff, a novel frame.work that constrains the difusion process, its essence is to establish a determin-istic mapping path between HR and LR, rather than the random noise disturbanceprocess oftraditional difusion models. Theoretical analysis demonstrates a 25%reduction in diffusion entropy in the residual space compared to pixel-space diffiusion, effectively suppressing irrelevant noise interference. The experimentalresults show that our method surpasses state-of-the-art models and generates re-sults with better fidelity. This work establishes a new low-rank constrained par-adigm for applying diffusion models to image reconstruction tasks, balancingstochastic generation with structural fidelity. Our code and model are publiclyavailable at https://github.com/continueyang/DeltaDiff .

cs.CV↗

Graph and Sequential Neural Networks in Session-based Recommendation: A Survey

Recent years have witnessed the remarkable success of recommendation systems (RSs) in alleviating the information overload problem. As a new paradigm of RSs, session-based recommendation (SR) specializes in users' short-term preference capture and aims to provide a more dynamic and timely recommendation based on the ongoing interacted actions. In this survey, we will give a comprehensive overview of the recent works on SR. First, we clarify the definitions of various SR tasks and introduce the characteristics of session-based recommendation against other recommendation tasks. Then, we summarize the existing methods in two categories: sequential neural network based methods and graph neural network (GNN) based methods. The standard frameworks and technical are also introduced. Finally, we discuss the challenges of SR and new research directions in this area.

cs.IR↗