arXiv subjects

Maoqi Liu

Publications and source records attributed to Maoqi Liu.

9 recordsLinked to original sources

arXiv2026

CoDe-LoRA: Mitigating the Orthogonality Dilemma in Continual Learning of LLMs via Knowledge Consolidation and Decoupling

Continual learning (CL) is essential for Large Language Models (LLMs) to sequentially adapt to evolving tasks. To mitigate catastrophic forgetting, recent advances implement low-rank adaptation with orthogonal projections (e.g., O-LoRA) to isolate task parameters. However, we reveal that such strict geometric constraints trigger an "Orthogonality Dilemma": rigid parameter isolation impedes the transfer and accumulation of shared representations across semantically related tasks. In this work, we propose a new replay-free method, called Consolidation and Decoupling LoRA (CoDe-LoRA), for CL of LLMs. CoDe-LoRA disentangles the learning process into Consolidating Universal Knowledge and Decoupling Task-Specific Knowledge. To achieve this, CoDe-LoRA leverages an adaptive null space projection mechanism and semantic routing to balance knowledge accumulation with task-specific adaptation. Experimental results across four backbones and three CL benchmarks show that CoDe-LoRA achieves the best average accuracy. Our code is available at https://github.com/Estrellajer/CoDe-LoRA.

cs.AI↗

arXiv2026

Scoring Higher, Answering Worse: Mitigating Reward Hacking in Rubric-Based RL via Protocol-Level Rubrics

Rubric-based reinforcement learning (Rubric-RL) trains language models where no verifier exists. A judge checks each criterion of a rubric, and the verdicts are aggregated into a reward, most often by a weighted sum. We show that this additive aggregation is the weak point. Under a sum, criteria compensate for one another: a policy that misses the one decision that matters can buy the points back with advice nobody asked for. On clinical consultation, such a policy scores higher and answers worse. Rubric coverage rises while appropriateness on held-out physician criteria falls below the untrained model. The medical criteria are not to blame. Grouped so that they must hold together, the same criteria, unchanged to the word, recover a third of the loss; shorter answers recover almost none. We therefore propose Protocol-level Rubrics (ProRubric), which keeps what the criteria ask for and changes how they are aggregated. It groups a checklist into a few protocol-level dimensions. A dimension counts only when all of its criteria hold and its failure clause does not fire. The grouping is done once, offline, and leaves the optimizer unchanged. ProRubric raises appropriateness by 10.8 points without losing coverage and has the best seven-benchmark average at both scales. Reward validity is set not only by what a rubric verifies, but by how it aggregates. Code is available at https://github.com/Estrellajer/ProRubric

cs.LG↗

arXiv2026

T2SPO: Trajectory-to-Step Policy Optimization for Agentic Reinforcement Learning

Reinforcement learning enables large language model (LLM) agents to learn multi-step behaviors through interaction with their environments. However, rewards in many interactive tasks reflect only the final outcome, providing limited guidance on which intermediate decisions advance the task. Successful training trajectories contain intermediate states that can provide supervision for subsequent interactions. We introduce Trajectory-to-Step Policy Optimization (T2SPO), a method that uses past interaction trajectories to provide step-level feedback for policy learning. T2SPO derives remaining-distance targets from successful trajectories and pairs them with representations of the states visited along the way. Conditioned on these examples, a pretrained TabPFN regressor estimates the remaining distance to success at each state of a new rollout. Changes in this distance estimate across consecutive states yield auxiliary credit for agent steps alongside task-level supervision. As training proceeds, newly completed trajectories refresh the estimator's context, incorporating new experience without updating its parameters. Experiments with 1.5B and 7B language models on ALFWorld and WebShop show that T2SPO consistently improves overall task success over GRPO.

cs.LG↗

arXiv2025

Knots: A Large-Scale Multi-Agent Enhanced Expert-Annotated Dataset and LLM Prompt Optimization for NOTAM Semantic Parsing

Notice to Air Missions (NOTAMs) serve as a critical channel for disseminating key flight safety information, yet their complex linguistic structures and implicit reasoning pose significant challenges for automated parsing. Existing research mainly focuses on surface-level tasks such as classification and named entity recognition, lacking deep semantic understanding. To address this gap, we propose NOTAM semantic parsing, a task emphasizing semantic inference and the integration of aviation domain knowledge to produce structured, inference-rich outputs. To support this task, we construct Knots (Knowledge and NOTAM Semantics), a high-quality dataset of 12,347 expert-annotated NOTAMs covering 194 Flight Information Regions, enhanced through a multi-agent collaborative framework for comprehensive field discovery. We systematically evaluate a wide range of prompt-engineering strategies and model-adaptation techniques, achieving substantial improvements in aviation text understanding and processing. Our experimental results demonstrate the effectiveness of the proposed approach and offer valuable insights for automated NOTAM analysis systems. Our code is available at: https://github.com/Estrellajer/Knots.

cs.CL↗

arXiv2025

NOTAM-Evolve: A Knowledge-Guided Self-Evolving Optimization Framework with LLMs for NOTAM Interpretation

Accurate interpretation of Notices to Airmen (NOTAMs) is critical for aviation safety, yet their condensed and cryptic language poses significant challenges to both manual and automated processing. Existing automated systems are typically limited to shallow parsing, failing to extract the actionable intelligence needed for operational decisions. We formalize the complete interpretation task as deep parsing, a dual-reasoning challenge requiring both dynamic knowledge grounding (linking the NOTAM to evolving real-world aeronautical data) and schema-based inference (applying static domain rules to deduce operational status). To tackle this challenge, we propose NOTAM-Evolve, a self-evolving framework that enables a large language model (LLM) to autonomously master complex NOTAM interpretation. Leveraging a knowledge graph-enhanced retrieval module for data grounding, the framework introduces a closed-loop learning process where the LLM progressively improves from its own outputs, minimizing the need for extensive human-annotated reasoning traces. In conjunction with this framework, we introduce a new benchmark dataset of 10,000 expert-annotated NOTAMs. Our experiments demonstrate that NOTAM-Evolve achieves a 30.4% absolute accuracy improvement over the base LLM, establishing a new state of the art on the task of structured NOTAM interpretation.

cs.CL↗

arXiv2025

Track Any Motions under Any Disturbances

A foundational humanoid motion tracker is expected to be able to track diverse, highly dynamic, and contact-rich motions. More importantly, it needs to operate stably in real-world scenarios against various dynamics disturbances, including terrains, external forces, and physical property changes for general practical use. To achieve this goal, we propose Any2Track (Track Any motions under Any disturbances), a two-stage RL framework to track various motions under multiple disturbances in the real world. Any2Track reformulates dynamics adaptability as an additional capability on top of basic action execution and consists of two key components: AnyTracker and AnyAdapter. AnyTracker is a general motion tracker with a series of careful designs to track various motions within a single policy. AnyAdapter is a history-informed adaptation module that endows the tracker with online dynamics adaptability to overcome the sim2real gap and multiple real-world disturbances. We deploy Any2Track on Unitree G1 hardware and achieve a successful sim2real transfer in a zero-shot manner. Any2Track performs exceptionally well in tracking various motions under multiple real-world disturbances.

cs.RO↗

arXiv2025

Simulation and Experimental Studies of DWDM Nonlinear Phase/Polarization/Power Crosstalk Between DFOS and Communication Channels in 27.6-Tb/s 800ZR Metro Network

We comprehensively analyze the fiber nonlinearity crosstalks between DAS and communication channels through numerical results and 40 x 800-Gb/s 90-km experimental demonstration. Our findings indicate that conventional pulse-based DAS is unsuitable for in-band DWDM coexistence system, whereas pulse-compression DAS shows negligible penalties with legacy coherent transceivers.

physics.optics↗

arXiv2024

Explainable Data-driven Share-of-choice Product Line Design Optimization

The share-of-choice (SOC) problem is a widely studied problem for product line design (PLD) where representative customers are sampled from a target population and the percentage of the ones who choose the offered products over outside options over the sample is maximized. The utility maximization framework captures individual choices. A significant challenge is that these utilities are not directly observable and must be estimated from other primitive data. Conjoint analysis is a commonly applied technique for generating such data, where sampled customers rate, rank, or choose between different product alternatives. With the responses, various methods, such as the hierarchical Bayesian method and polyhedral estimation, are employed to estimate the utilities. However, this " estimate-then-optimize" procedure disconnects the decision-making process from the primitive data and thus cannot attribute the PLD decision to the unresolved survey data. To fill the gap, we integrate the polyhedral estimation method into the PLD optimization, which constructs a polyhedron set containing all utilities consistent with the responses. Specifically, we proposed a robust model that maximizes the share-of-choice calculated by the worst-case utility over the polyhedron. The model brings threefold explainability to the PLD problem. First, it enables us to explore the influence of both estimation error and the number of sampled customers through an out-of-sample performance guarantee of the proposed model. Secondly, the linearly tractable equivalent reformulation constructed by the duality allows us to attribute the optimal product line to the survey data through the dual variables. Third, inspired by the fact that the survey questions are associated with columns in the dual reformulation, we propose a column generation approach to identify new questions that most effectively improve the model's objective.

math.OC↗

arXiv2024

Achieving Robust Data-driven Contextual Decision Making in a Data Augmentation Way

This paper focuses on the contextual optimization problem where a decision is subject to some uncertain parameters and covariates that have some predictive power on those parameters are available before the decision is made. More specifically, we focus on solving the Wasserstein-distance-based distributionally robust optimization (DRO) model for the problem, which maximizes the worst-case expected objective over an uncertainty set including all distributions closed enough to a nominal distribution with respect to the Wasserstein distance. We develop a stochastic gradient descent algorithm based on the idea of data augmentation to solve the model efficiently. The algorithm iteratively a) does a bootstrapping sample from the nominal distribution; b) perturbs the adversarially and c) updates decisions. Accordingly, the computational time of the algorithm is only determined by the number of iterations and the complexity of computing the gradient of a single sample. Except for efficiently solving the model, the algorithm provide additional advantages that the proposed algorithm can cope with any nominal distributions and therefore is extendable to solve the problem in an online setting. We also prove that the algorithm converges to the optimal solution of the DRO model at a rate of a $O(1/\sqrt{T})$, where $T$ is the number of iterations of bootstrapping. Consequently, the performance guarantee of the algorithm is that of the DRO model plus $O(1/\sqrt{T})$. Through extensive numerical experiments, we demonstrate the superior performance of the proposed algorithm to several benchmarks.

math.OC↗