arXiv subjects

Huiyan Xue

Publications and source records attributed to Huiyan Xue.

6 recordsLinked to original sources

arXiv2026

Representation Finetuning for Continual Learning

The world is inherently dynamic, and continual learning aims to enable models to adapt to ever-evolving data streams. While pre-trained models have shown powerful performance in continual learning, they still require finetuning to adapt effectively to downstream tasks. However, prevailing Parameter-Efficient Fine-Tuning (PEFT) methods operate through empirical, black-box optimization at the weight level. These approaches lack explicit control over representation drift, leading to sensitivity to domain shifts and catastrophic forgetting in continual learning scenarios. In this work, we introduce Continual Representation Learning (CoRe), a novel framework that for the first time shifts the finetuning paradigm from weight space to representation space. Unlike conventional methods, CoRe performs task-specific interventions within a low-rank linear subspace of hidden representations, adopting a learning process with explicit objectives, which ensures stability for past tasks while maintaining plasticity for new ones. By constraining updates to a low-rank subspace, CoRe achieves exceptional parameter efficiency. Extensive experiments across multiple continual learning benchmarks demonstrate that CoRe not only preserves parameter efficiency but also significantly outperforms existing state-of-the-art methods. Our work introduces representation finetuning as a new, more effective and interpretable paradigm for continual learning.

cs.LG↗

arXiv2026

Key-Value Pair-Free Continual Learner via Task-Specific Prompt-Prototype

Continual learning aims to enable models to acquire new knowledge while retaining previously learned information. Prompt-based methods have shown remarkable performance in this domain; however, they typically rely on key-value pairing, which can introduce inter-task interference and hinder scalability. To overcome these limitations, we propose a novel approach employing task-specific Prompt-Prototype (ProP), thereby eliminating the need for key-value pairs. In our method, task-specific prompts facilitate more effective feature learning for the current task, while corresponding prototypes capture the representative features of the input. During inference, predictions are generated by binding each task-specific prompt with its associated prototype. Additionally, we introduce regularization constraints during prompt initialization to penalize excessively large values, thereby enhancing stability. Experiments on several widely used datasets demonstrate the effectiveness of the proposed method. In contrast to mainstream prompt-based approaches, our framework removes the dependency on key-value pairs, offering a fresh perspective for future continual learning research.

cs.AI↗

arXiv2025

Distillation-Guided Structural Transfer for Continual Learning Beyond Sparse Distributed Memory

Sparse neural systems are gaining traction for efficient continual learning due to their modularity and low interference. Architectures such as Sparse Distributed Memory Multi-Layer Perceptrons (SDMLP) construct task-specific subnetworks via Top-K activation and have shown resilience against catastrophic forgetting. However, their rigid modularity limits cross-task knowledge reuse and leads to performance degradation under high sparsity. We propose Selective Subnetwork Distillation (SSD), a structurally guided continual learning framework that treats distillation not as a regularizer but as a topology-aligned information conduit. SSD identifies neurons with high activation frequency and selectively distills knowledge within previous Top-K subnetworks and output logits, without requiring replay or task labels. This enables structural realignment while preserving sparse modularity. Experiments on Split CIFAR-10, CIFAR-100, and MNIST demonstrate that SSD improves accuracy, retention, and representation coverage, offering a structurally grounded solution for sparse continual learning.

cs.LG↗

arXiv2014

A classification of volume preserving generating forms in R^3

In earlier work, Lomeli and Meiss used a generalization of the symplectic approach to study volume preserving generating differential forms. In particular, for the $\mathbb{R}^3$ case, the first to differ from the symplectic case, they derived thirty-six one-forms that generate exact volume preserving maps. Xue and Zanna had studied these differential forms in connection with the numerical solution of divergence-free differential equations: can such forms be used to devise new volume preserving integrators or to further understand existing ones? As a partial answer to this question, Xue and Zanna showed how six of the generating volume form were naturally associated to consistent, first order, volume preserving numerical integrators. In this paper, we investigate and classify the remaining cases. The main result is the reduction of the thirty-six cases to five essentially different cases, up to variable relabeling and adjunction. We classify these five cases, identifying two novel classes and associating the other three to volume preserving vector fields under a Hamiltonian or Lagrangian representation. We demonstrate how these generating form lead to consistent volume preserving schemes for volume preserving vector fields in $\mathbb{R}^3$.

math.NA↗

arXiv2013

High order volume preserving integrators for three kinds of divergence-free vector fields via commutator

In this paper, we focus on the construction of high order volume preserving in- tegrators for divergence-free vector fields: the monomial basis, the exponential basis and tensor product of the monomial and the exponential basis. We first prove that the commutators of elementary divergence-free vector fields (EDFVF) of those three kinds are still divergence-free vector fields of the same kind. Assuming then there is only diagonal part of divergence-free vector field of the monomial basis, for those three kinds of divergence-free vector fields, we construct high order volume-preserving inte- grators using the multi-commutators for EDFVFs. Moreover, we consider the ordering of the EDFVFs and their commutators to reduce the error of the schemes, showing by numerical tests that the strategy in [9] works very well.

math.NA↗

arXiv2012

Explicit Volume-Preserving Splitting Methods for Polynomial Divergence-Free Vector Fields

We present new, explicit, volume-preserving vector fields for polynomial divergence-free vector fields of arbitrary degree (both positive and negative). The main idea is to decompose the divergence polynomial by means of an appropriate basis for polynomials: the monomial basis. For each monomial basis function, the split fields are then identified by collecting the appropriate terms in the vector field so that each split vector field is divergence free. We show that each split field can be integrated exactly by analytical methods. Thus, the composition yields a volume preserving numerical method. Our numerical tests indicate that the methods compare favorably to standard integrators both in the quality of the numerical solution and the computational effort.

math.NA↗