arXiv subjects

Zhen-Yu Zhang

Publications and source records attributed to Zhen-Yu Zhang.

10 recordsLinked to original sources

arXiv2026

TreeLoRA: Efficient Continual Learning via Layer-Wise LoRAs Guided by a Hierarchical Gradient-Similarity Tree

Many real-world applications collect data in a streaming environment, where learning tasks are encountered sequentially. This necessitates continual learning (CL) to update models online, enabling adaptation to new tasks while preserving past knowledge to prevent catastrophic forgetting. Nowadays, with the flourish of large pre-trained models (LPMs), efficiency has become increasingly critical for CL, due to their substantial computational demands and growing parameter sizes. In this paper, we introduce TreeLoRA (K-D Tree of Low-Rank Adapters), a novel approach that constructs layer-wise adapters by leveraging hierarchical gradient similarity to enable efficient CL, particularly for LPMs. To reduce the computational burden of task similarity estimation, we employ bandit techniques to develop an algorithm based on lower confidence bounds to efficiently explore the task structure. Furthermore, we use sparse gradient updates to facilitate parameter optimization, making the approach better suited for LPMs. Theoretical analysis is provided to justify the rationale behind our approach, and experiments on both vision transformers (ViTs) and large language models (LLMs) demonstrate the effectiveness and efficiency of our approach across various domains, including vision and natural language processing tasks.

cs.LG↗

arXiv2026

Data-dependent Exploration for Online Reinforcement Learning from Human Feedback

Online reinforcement learning from human feedback (RLHF) has emerged as a promising paradigm for aligning large language models (LLMs) by continuously collecting new preference feedback during training. A foundational challenge in this setting is exploration, which requires algorithms that enable the LLMs to generate informative comparisons that improve sample-efficiency in online RLHF. Existing exploration strategies often derive bonuses via on-policy expectations, which are difficult to estimate reliably from the limited historical preference data available during training; as a result, the policy can prematurely down-weight under-explored regions that may contain high-value behaviors. In this paper, we propose data-dependent exploration for preference optimization (DEPO), a simple and scalable method that leverages historical data to construct an extra uncertainty bonus for high-uncertainty regions, encouraging exploration toward potentially high-value data. Theoretically, we provide a data-dependent regret bound for the proposed algorithm, showing that it adapts to the hardness of the learning task itself and can be tighter than worst-case bounds in practice. Empirically, the proposed method consistently outperforms strong baselines across benchmarks, demonstrating improved sample efficiency.

cs.LG↗

arXiv2026

Selective Ensemble Based on Preference-Directed Multi-Objective Bandits

Selective ensemble for modern machine learning systems requires choosing promising model candidates under limited evaluation budgets, while downstream tasks often specify only partial preferences over capabilities such as accuracy, robustness, and reasoning. This setting naturally gives rise to a sequential decision problem under partially specified linear preferences. We formalize it as preference-directed multi-objective bandits (PDMOB), where admissible trade-offs are represented by a polyhedral preference cone. Based on this formulation, we introduce Pareto $C$-optimality, which recovers standard Pareto optimality and single-weight scalarization as special cases. We then propose the preference-directed upper confidence bound (PrefUCB) algorithm, which maintains directional confidence intervals to guide exploration. We analyze both indicator-based and gap-weighted regret, and establish instance-dependent logarithmic bounds for both criteria, recovering the optimal logarithmic dependence on the horizon $T$ in classical special cases. Experiments on large pre-trained model selective ensemble tasks and online asset allocation under institutional mandates validate the efficacy of our method.

cs.LG↗

arXiv2025

In-context Demonstration Matters: On Prompt Optimization for Pseudo-Supervision Refinement

Large language models (LLMs) have achieved great success across diverse tasks, and fine-tuning is sometimes needed to further enhance generation quality. Most existing methods rely on human supervision or parameter retraining, both of which are costly in terms of data collection and computational resources. To handle these challenges, a direct solution is to generate ``high-confidence'' data from unsupervised downstream tasks and use them for in-context prompting or prompt optimization to refine the pseudo-supervision. However, relying solely on such data may lead to overfitting. In this paper, we leverage the in-context learning (ICL) abilities of LLMs and propose a novel approach, pseudo-supervised demonstrations aligned prompt optimization (PAPO) algorithm, which jointly refines both the prompt and the overall pseudo-supervision. The proposed learning objective ensures that the optimized prompt guides the LLM to generate consistent responses for a given input when pseudo-supervised data from the downstream task are used as demonstrations, enabling refinement over the entire pseudo-supervision. The prompt is optimized by translating gradient signals into textual critiques, which serve as feedback to iteratively refine the prompt and model responses. Theoretical analysis in a simplified classification setting shows that the refined pseudo-supervision exhibits a geometric clustering structure, helping to mitigate overfitting. Experiments on question answering, natural language inference benchmarks, and a real-world molecule optimization task, show the effectiveness of the proposed algorithm.

cs.CL↗

arXiv2024

The pole structures of the $X(1840)/X(1835)$ and the $X(1880)$

Whether the $N\bar{N}$ interaction could form a state or not is a long standing question, even before the observation of the $p\bar{p}$ threshold enhancement in 2003. The recent high statistic measurement in the $J/ψ\to γ3(π^+π^-)$ channel would provide a good opportunity to probe the nature of the peak structures around the $p\bar{p}$ threshold in various processes. By constructing the $N\bar{N}$ interaction respecting chiral symmetry, we extract the pole positions by fitting the $p\bar{p}$ and $3(π^+π^-)$ invariant mass distributions of the $J/ψ\to γp \bar p$ and $J/ψ\to γ3(π^+π^-)$ processes. The threshold enhancement in the $p\bar{p}$ invariant mass distribution is from the pole on the third Riemann sheet, which more couples to the isospin triplet channel. The broader structure in the $3(π^+π^-)$ invariant mass comes from the pole on the physical Riemann sheet, which more couples to the isospin singlet channel. Furthermore, the large compositeness indicates that there should exit $p\bar{p}$ resonance based on the current experimental data. In addition, we also see a clear threshold enhancement in the $n\bar{n}$ channel, but not as significant as that in $p\bar{p}$ channel, which is useful and compared with further experimental measurement.

hep-ph↗

arXiv2024

Generating Chain-of-Thoughts with a Pairwise-Comparison Approach to Searching for the Most Promising Intermediate Thought

To improve the ability of the large language model (LLMs) to tackle complex reasoning problems, chain-of-thoughts (CoT) methods were proposed to guide LLMs to reason step-by-step, enabling problem solving from simple to complex. State-of-the-art methods for generating such a chain involve interactive collaboration, where the learner generates candidate intermediate thoughts, evaluated by the LLM, guiding the generation of subsequent thoughts. However, a widespread yet understudied problem is that the evaluation from the LLM is typically noisy and unreliable, potentially misleading the generation process in selecting promising intermediate thoughts. In this paper, motivated by Vapnik's principle, we use pairwise-comparison evaluation instead of point-wise scoring to search for promising intermediate thoughts with the noisy feedback from the LLM. In each round, we randomly pair intermediate thoughts and directly prompt the LLM to select the more promising one from each pair, allowing us to identify the most promising thoughts through an iterative process. To further alleviate the noise in the comparison, we incorporate techniques from ensemble learning and dueling bandits, proposing two variants of the algorithm. Experiments on three real-world tasks demonstrate the effectiveness of our proposed algorithm and verify the rationale of the pairwise comparison mechanism.

cs.LG↗

arXiv2024

Adapting to Continuous Covariate Shift via Online Density Ratio Estimation

Dealing with distribution shifts is one of the central challenges for modern machine learning. One fundamental situation is the covariate shift, where the input distributions of data change from training to testing stages while the input-conditional output distribution remains unchanged. In this paper, we initiate the study of a more challenging scenario -- continuous covariate shift -- in which the test data appear sequentially, and their distributions can shift continuously. Our goal is to adaptively train the predictor such that its prediction risk accumulated over time can be minimized. Starting with the importance-weighted learning, we show the method works effectively if the time-varying density ratios of test and train inputs can be accurately estimated. However, existing density ratio estimation methods would fail due to data scarcity at each time step. To this end, we propose an online method that can appropriately reuse historical information. Our density ratio estimation method is proven to perform well by enjoying a dynamic regret bound, which finally leads to an excess risk guarantee for the predictor. Empirical results also validate the effectiveness.

cs.LG↗

arXiv2022

Early Abnormal Detection of Sewage Pipe Network: Bagging of Various Abnormal Detection Algorithms

Abnormalities of the sewage pipe network will affect the normal operation of the whole city. Therefore, it is important to detect the abnormalities early. This paper propose an early abnormal-detection method. The abnormalities are detected by using the conventional algorithms, such as isolation forest algorithm, two innovations are given: (1) The current and historical data measured by the sensors placed in the sewage pipe network (such as ultrasonic Doppler flowmeter) are taken as the overall dataset, and then the general dataset is detected by using the conventional anomaly detection method to diagnose the anomaly of the data. The anomaly refers to the sample different from the others samples in the whole dataset. Because the definition of anomaly is not through the algorithm, but the whole dataset, the construction of the whole dataset is the key to propose the early abnormal-detection algorithms. (2) A bagging strategy for a variety of conventional anomaly detection algorithms is proposed to achieve the early detection of anomalies with the high precision and recall. The results show that this method can achieve the early anomaly detection with the highest precision of 98.21%, the recall rate 63.58% and F1-score of 0.774.

cs.LG↗

arXiv2017

Event generators for $η/η^{\prime}$ decays at BESIII

The light unflavoured meson $η/η^{\prime}$ decays are valuable for testing non-perturbative quantum chromodynamics and exploring new physics beyond the Standard Model. This paper describes a series of event generators, including $η/η^{\prime}\toγl^{+}l^{-}$, $η/η^{\prime}\toγπ^{+}π^{-}$, $η^{\prime}\toωe^{+}e^{-}$, $η\toπ^{+}π^{-}π^{0}$, $η/η^{\prime}\toπ^{0}π^{0}π^{0}$, $η^{\prime}\toηππ$ and $η^{\prime}\toπ^{+}π^{-}π^{+}π^{-}/π^{+}π^{-}π^{0}π^{0}$, which have been developed for investigating $η/η^\prime$ decay dynamics. For most of these generators, their usability has been validated in BESIII analyses for determining the detection efficiency, and background studies. The consistency between data and Monte Carlo shows that these generators work well in the BESIII simulation, and will also be useful for ongoing BESIII analyses and other experiments for studying $η/η^\prime$ physics.

hep-ex↗

arXiv2010

Equivalent Circuit Description of Non-compensated n-p Codoped TiO2 as Intermediate Band Solar Cells

The novel concept of non-compensated n-p codoping has made it possible to create tunable intermediate bands in the intrinsic band gap of TiO2, making the codoped TiO2 a promising material for developing intermediate band solar cells (IBSCs). Here we investigate the quantum efficiency of such IBSCs within two scenarios - with and without current extracted from the extended intermediate band. Using the ideal equivalent circuit model, we find that the maximum efficiency of 57% in the first scenario and 53% in the second are both much higher than the Shockley-Queisser limit from single gap solar cells. We also obtain various key quantities of the circuits, a useful step in realistic development of TiO2 based solar cells invoking device integration. These equivalent circuit results are also compared with the efficiencies obtained directly from consideration of electron transition between the energy bands, and both approaches reveal the intriguing existence of double peaks in the maximum quantum efficiency as a function of the relative location of IBs.

cond-mat.mtrl-sci↗