Searcharxiv⌕ Search

arXiv subjects

Xufeng Liu

Publications and source records attributed to Xufeng Liu.

10 recordsLinked to original sources
arXiv2026

Sparse Attention Is Matrix Approximation, Not Choosing from a Bag of Values

Large Language Models (LLMs) achieve strong performance across many domains, but their efficiency is limited by the quadratic cost of attention with respect to prompt length. Sparse attention reduces this cost by retaining only a small fraction of query-key interactions to approximate the full attention matrix. However, existing methods are trapped in a mathematically wrong view: they simply keep large scalar entries or high-mass regions of the attention matrix. This treats the attention matrix as a bag of values, ignoring that it is used as a structured matrix whose entries jointly determine the attention output through multiplication with value vectors. We argue that this is the core conceptual issue: sparse attention should be formulated as matrix approximation, not as blindly choosing the largest values from a bag of entries. Based on this view, we propose Matrix Approximation Sparse Attention (MASA). MASA replaces raw attention-mass ranking with a closed-form score that measures how much each sparse unit reduces matrix-product approximation error. As a theory-grounded plug-in correction, MASA can be added to existing sparse attention frameworks without changing their sparse kernels or budgets. Extensive experiments across multiple sparse attention methods, benchmarks, and LLM backbones show consistent accuracy gains, supporting both MASA and the matrix-approximation view of sparse attention.

cs.CL↗
arXiv2026

Beyond Prediction: A Component-Level Study of Concept Recovery in In-Context Learning

In-context learning (ICL) enables language models to make accurate predictions from demonstrations, but prediction success alone does not reveal the extent to which the underlying concept has been recovered. Existing evaluations typically assess recovery at the level of the whole concept, making it difficult to identify which components are recovered and to separate model limitations from ambiguity in the demonstrations. We introduce a component-level framework for concept recovery that decomposes concepts into their defining components and uses controlled structural interventions to evaluate both component-level and whole-concept recovery under identifiable demonstrations. Across four instruction-tuned LLMs and three composition forms, we find that models often achieve high prediction accuracy despite poor whole-concept recovery, with systematic over-specification of relevant attributes and level constraints. Individual structural components are generally insufficient for whole-concept recovery, whereas jointly specifying complementary components, particularly relevant attributes and their level constraints, substantially improves recovery. However, this improvement is fragile: a single plausible but unnecessary level constraint can substantially disrupt recovery. Finally, joint supervision of recovery and implicit prediction substantially improves recovery while preserving prediction performance, with the learned capability transferring across composition forms. Overall, our component-level analysis provides a more diagnostic view of what models recover from demonstrations than the existing holistic view.

cs.LG↗
arXiv2026

AnchorMixGAN: Anchor-Aligned Generative Semi-Supervision for DDoS Detection in Cloud-Integrated IoT Networks

Detecting distributed denial-of-service (DDoS) attacks in cloud-integrated IoT networks is difficult when labeled traffic is scarce. Generative semi-supervised learning can supplement the available training data, but prediction shifts induced by synthetic views may affect the targets assigned to real unlabeled flows. We propose AnchorMixGAN, a generative semi-supervised framework that addresses this problem through anchor-aligned target construction. Its Anchor-MAS module treats each real unlabeled flow as an anchor and creates alternative views by replacing one field group at a time with values from generated traffic. A frozen reference classifier predicts the anchor and its views; averaging and sharpening these predictions produces a soft target for the original flow. The flow and its target are then mixed with a labeled example using MixUp, allowing the detector to learn from both the original labeled records and the mixed examples. We analyze how reference-classifier error, view construction, and sharpening affect the target, and derive a bound on the resulting change in cross-entropy at a fixed detector prediction. At the reported 90% training setting with 20% of the training records labeled, AnchorMixGAN attains accuracies of 97.3%, 97.4%, and 96.5% on NSLKDD, BoT-IoT, and CICIoT2023, respectively, exceeding the corresponding MixGAN results by 1.6, 1.0, and 4.4 percentage points.

cs.LG↗
arXiv2025

Pilot selection in the era of Virtual reality: algorithms for accurate and interpretable machine learning models

With the rapid growth of the aviation industry, there is a need for a large number of flight crew. How to select the right pilots in a cost-efficient manner has become an important research question. In the current study, twenty-three pilots were recruited from China Eastern Airlines, and 23 novices were from the community of Tsinghua University. A novel approach incorporating machine learning and virtual reality technology was applied to distinguish features between these participants with different flight skills. Results indicate that SVM with the MIC feature selection method consistently achieved the highest prediction performance on all metrics with an Accuracy of 0.93, an AUC of 0.96, and an F1 of 0.93, which outperforms four other classifier algorithms and two other feature selection methods. From the perspective of feature selection methods, the MIC method can select features with a nonlinear relationship to sampling labels, instead of a simple filter-out. Our new implementation of the SVM + MIC algorithm outperforms all existing pilot selection algorithms and perhaps provides the first implementation based on eye tracking and flight dynamics data. This study's VR simulation platforms and algorithms can be used for pilot selection and training.

cs.LG↗
arXiv2025

RISE: Radius of Influence based Subgraph Extraction for 3D Molecular Graph Explanation

3D Geometric Graph Neural Networks (GNNs) have emerged as transformative tools for modeling molecular data. Despite their predictive power, these models often suffer from limited interpretability, raising concerns for scientific applications that require reliable and transparent insights. While existing methods have primarily focused on explaining molecular substructures in 2D GNNs, the transition to 3D GNNs introduces unique challenges, such as handling the implicit dense edge structures created by a cut-off radius. To tackle this, we introduce a novel explanation method specifically designed for 3D GNNs, which localizes the explanation to the immediate neighborhood of each node within the 3D space. Each node is assigned an radius of influence, defining the localized region within which message passing captures spatial and structural interactions crucial for the model's predictions. This method leverages the spatial and geometric characteristics inherent in 3D graphs. By constraining the subgraph to a localized radius of influence, the approach not only enhances interpretability but also aligns with the physical and structural dependencies typical of 3D graph applications, such as molecular learning.

cs.LG↗
arXiv2025

M-ary Precomputation-Based Accelerated Scalar Multiplication Algorithms for Enhanced Elliptic Curve Cryptography

Efficient scalar multiplication is critical for enhancing the performance of elliptic curve cryptography (ECC), especially in applications requiring large-scale or real-time cryptographic operations. This paper proposes an M-ary precomputation-based scalar multiplication algorithm, aiming to optimize both computational efficiency and memory usage. The method reduces the time complexity from $Θ(Q \log p)$ to $Θ\left(\frac{Q \log p}{\log Q}\right)$ and achieves a memory complexity of $Θ\left(\frac{Q \log p}{\log^2 Q}\right)$. Experiments on ElGamal encryption and NS3-based communication simulations validate its effectiveness. On secp256k1, the proposed method achieves up to a 59\% reduction in encryption time and 30\% memory savings. In network simulations, the binary-optimized variant reduces communication time by 22.1\% on secp384r1 and simulation time by 25.4\% on secp521r1. The results demonstrate the scalability, efficiency, and practical applicability of the proposed algorithm. The source code will be publicly released upon acceptance.

cs.CR↗
arXiv2023

A Criterion for Unentanglement of Quantum States

The concept of entanglement is at the core of the theory of quantum information. In this paper a criterion for unentanglement of quantum states is proposed and proved. This criterion is natural, practical and easy to check.

quant-ph↗
arXiv2000

Representations of $U_ε^{res}(sl_2)$ via Restricted q-Fock Spaces

Two restricted $C[q,q^{-1}]-$forms of the well known q-boson algebra are introduced and the corresponding restricted q-Fock spaces defined. All of the irreducible highest weight representations, including the infinite dimensional ones, of $U_ε^{res}(sl_2)$ of type 1 are constructed through the restricted q-Fock spaces.

math.QA↗