arXiv subjects

Chaofan Pan

Publications and source records attributed to Chaofan Pan.

8 recordsLinked to original sources

arXiv2026

TOUR: A Trajectory-Level Unlearning Benchmark for Offline Reinforcement Learning

Offline Reinforcement Learning (RL) agents are trained on fixed behavioral trajectories, which makes trajectory-level deletion important when selected data must be removed after training. Evaluating such deletion is difficult because a lower membership score can reflect trajectory removal, residual memorization visible to another attack, or policy collapse that destroys useful behavior. We introduce Trajectory-level memOrization and Unlearning in offline RL (TOUR), a benchmark that combines trajectory-level partitioning, matched non-member controls, retraining references, retained-performance anchors, and multi-attack privacy auditing. Across D4RL locomotion experiments and an exploratory AntMaze extension, TOUR shows that common deletion baselines have environment-dependent privacy-utility behavior. Retraining and fine-tuning often provide stronger retained-utility references than uniform GA+Refit, while TrajDeleter remains a useful comparator but is not uniformly stronger under the same audit. Reference-model, threshold, deviation, equivalence, action-error, representation-based, and query-limited attacks further show that a single likelihood-based membership score can overstate deletion quality. In the evaluated settings, conclusions about offline RL unlearning are therefore not stable under single-score auditing. They depend on matched non-member construction, retraining-relative calibration, attack family, retained utility, and explicit scope for diagnostic architecture or component-level evidence.

cs.LG↗

arXiv2026

Regime-Adaptive Continual Learning for Portfolio Management

Financial markets are inherently non-stationary, exhibiting frequent regime shifts and structural changes that render traditional Portfolio Management (PM) approaches ineffective. Existing remedies, such as rolling-window retraining and naive online fine-tuning, are hindered by high computational costs and insufficient knowledge utilization, respectively, resulting in low returns and limited adaptability. Continual learning (CL) offers a promising paradigm by enabling trading agents to accumulate and transfer knowledge across sequential tasks. In this paper, we propose \textbf{Re}gime-aware \textbf{C}ontinual \textbf{A}daptive \textbf{P}ortfolio management (\textbf{ReCAP}), a novel framework that integrates CL into PM to address the challenges of dynamic financial environments. ReCAP employs an adaptive regime detection module to segment historical market data into variable-length regimes, enabling regime-specific learning of policy vectors and the construction of a policy library. During continual trading, a regime-gate module adaptively combines policy vectors from the library based on the current market state, facilitating rapid adaptation to newly detected regimes. Only the regime-gate and the current regime's policy vector are continually updated to preserve useful knowledge effectively. Extensive experiments on five real-world datasets demonstrate that ReCAP consistently outperforms popular baselines, achieving superior returns in long-term investment horizons and rapid adaptation to regime shifts.

q-fin.PM↗

arXiv2025

A Survey of Continual Reinforcement Learning

Reinforcement Learning (RL) is an important machine learning paradigm for solving sequential decision-making problems. Recent years have witnessed remarkable progress in this field due to the rapid development of deep neural networks. However, the success of RL currently relies on extensive training data and computational resources. In addition, RL's limited ability to generalize across tasks restricts its applicability in dynamic and real-world environments. With the arisen of Continual Learning (CL), Continual Reinforcement Learning (CRL) has emerged as a promising research direction to address these limitations by enabling agents to learn continuously, adapt to new tasks, and retain previously acquired knowledge. In this survey, we provide a comprehensive examination of CRL, focusing on its core concepts, challenges, and methodologies. Firstly, we conduct a detailed review of existing works, organizing and analyzing their metrics, tasks, benchmarks, and scenario settings. Secondly, we propose a new taxonomy of CRL methods, categorizing them into four types from the perspective of knowledge storage and/or transfer. Finally, our analysis highlights the unique challenges of CRL and provides practical insights into future directions.

cs.LG↗

arXiv2025

Action-Adaptive Continual Learning: Enabling Policy Generalization under Dynamic Action Spaces

Continual Learning (CL) is a powerful tool that enables agents to learn a sequence of tasks, accumulating knowledge learned in the past and using it for problem-solving or future task learning. However, existing CL methods often assume that the agent's capabilities remain static within dynamic environments, which doesn't reflect real-world scenarios where capabilities dynamically change. This paper introduces a new and realistic problem: Continual Learning with Dynamic Capabilities (CL-DC), posing a significant challenge for CL agents: How can policy generalization across different action spaces be achieved? Inspired by the cortical functions, we propose an Action-Adaptive Continual Learning framework (AACL) to address this challenge. Our framework decouples the agent's policy from the specific action space by building an action representation space. For a new action space, the encoder-decoder of action representations is adaptively fine-tuned to maintain a balance between stability and plasticity. Furthermore, we release a benchmark based on three environments to validate the effectiveness of methods for CL-DC. Experimental results demonstrate that our framework outperforms popular methods by generalizing the policy across action spaces.

cs.LG↗

arXiv2024

Multi-Granularity Open Intent Classification via Adaptive Granular-Ball Decision Boundary

Open intent classification is critical for the development of dialogue systems, aiming to accurately classify known intents into their corresponding classes while identifying unknown intents. Prior boundary-based methods assumed known intents fit within compact spherical regions, focusing on coarse-grained representation and precise spherical decision boundaries. However, these assumptions are often violated in practical scenarios, making it difficult to distinguish known intent classes from unknowns using a single spherical boundary. To tackle these issues, we propose a Multi-granularity Open intent classification method via adaptive Granular-Ball decision boundary (MOGB). Our MOGB method consists of two modules: representation learning and decision boundary acquiring. To effectively represent the intent distribution, we design a hierarchical representation learning method. This involves iteratively alternating between adaptive granular-ball clustering and nearest sub-centroid classification to capture fine-grained semantic structures within known intent classes. Furthermore, multi-granularity decision boundaries are constructed for open intent classification by employing granular-balls with varying centroids and radii. Extensive experiments conducted on three public datasets demonstrate the effectiveness of our proposed method.

cs.CL↗

arXiv2024

Multi-granularity Knowledge Transfer for Continual Reinforcement Learning

Continual reinforcement learning (CRL) empowers RL agents with the ability to learn a sequence of tasks, accumulating knowledge learned in the past and using the knowledge for problemsolving or future task learning. However, existing methods often focus on transferring fine-grained knowledge across similar tasks, which neglects the multi-granularity structure of human cognitive control, resulting in insufficient knowledge transfer across diverse tasks. To enhance coarse-grained knowledge transfer, we propose a novel framework called MT-Core (as shorthand for Multi-granularity knowledge Transfer for Continual reinforcement learning). MT-Core has a key characteristic of multi-granularity policy learning: 1) a coarsegrained policy formulation for utilizing the powerful reasoning ability of the large language model (LLM) to set goals, and 2) a fine-grained policy learning through RL which is oriented by the goals. We also construct a new policy library (knowledge base) to store policies that can be retrieved for multi-granularity knowledge transfer. Experimental results demonstrate the superiority of the proposed MT-Core in handling diverse CRL tasks versus popular baselines.

cs.LG↗

arXiv2023

Exponential trichotomy and global linearization of non-autonomous differential equations

Hartman-Grobman theorem was initially extended to the non-autonomous cases by Palmer. Usually, dichotomy is an essential condition of Palmer's linearization theorem. Is Palmer's linearization theorem valid for the systems with trichotomy? In this paper, we obtain new versions of the linearization theorem if linear system admits exponential trichotomy on $\mathbb{R}$. { Furthermore, the equivalent function $\mathscr H(t,x)$ and its inverse $\mathscr L(t,y)$ of our linearization theorems are H\"{o}lder continuous}. In addition, if a system is periodic, we find the equivalent function $\mathscr H(t,x)$ and its inverse $\mathscr L(t,y)$ of our linearization theorems do not have periodicity or asymptotical periodicity. To the best of our knowledge, this is the first paper studying the linearization with exponential trichotomy.

math.CA↗

arXiv2022

A Hartman-Grobman theorem for algebraic dichotomies

Algebraic dichotomy is a generalization of an exponential dichotomy (Lin, JDE2009). This paper gives a version of Hartman-Grobman linearization theorem assuming that linear system admits an algebraic dichotomy, which generalizes the Palmer's linearization theorem. Besides, we prove that the homeomorphism in the linearization theorem (and has a H\"{o}lder continuous inverse). Comparing with exponential dichotomy, algebraic dichotomy is more complicate. The exponential dichotomy leads to the estimates $\int_{-\infty}^{t}e^{-\alpha(t-s)}ds$ and $\int_{t}^{+\infty}e^{-\alpha(s-t)}ds$ which are convergent. However, the algebraic dichotomy will leads us to $\int_{-\infty}^{t}\left(\frac{\mu(t)}{\mu(s)}\right)^{-\alpha}ds$ or $\int_{t}^{+\infty}\left(\frac{\mu(s)}{\mu(t)}\right)^{-\alpha}ds$, whose the convergence is unknown in the sense of Riemann.

math.CA↗