SearcharxivSearch

arXiv subjects

Huimin Han

Publications and source records attributed to Huimin Han.

3 recordsLinked to original sources

K/V-Cache Interventions Dissociate Representation Alignment from Persona Expression in Decoder-Only Language Models

We study K/V-cache interventions -- transplanting a target-conditioned K/V trajectory into a source-persona generation -- as a structured surface for persona control in decoder-only language models. Across 13 intervention configurations applied to Llama-3.1-8B for a fixed source-to-target persona pair, we report two consistent dissociations between representation-level alignment and behavioral expression, plus a common failure under position perturbations. First, all layer-band K/V replacements (early, mid, late) achieve strong local V-space alignment (V-gap 0.91, 0.89, 0.84), but only mid-layer replacement (layers 9-20) combines substantial target-marker expression with preserved lexical diversity. Second, full and mid-layer replacement induce comparable alignment (V-gap 0.94 vs. 0.89) yet produce different lexical-diversity profiles (TTR 0.65 vs. 0.77). Third, position perturbations (lag and shuffle) apply distinct operations yet uniformly suppress target-persona expression -- a common behavioral failure rather than a strict dissociation. Representation-level similarity metrics alone are thus not sufficient predictors of downstream persona expression in the regimes we study; the K/V cache emerges as a controllable but structurally constrained intervention surface. Because the transplanted trajectory carries the target's own generated token history, we characterize the intervention as trajectory-level transplantation rather than isolated persona-representation injection; a same-token-sequence control, decoding an identical token sequence under source vs. target conditioning, reproduces the sign and layer localization of the L28 representational shift, indicating the shift is not explained solely by imported token history. These findings characterize representation-behavior dissociation in a high-signal setting rather than establishing universality across models or persona pairs.

cs.CL

SPEAR: Code-Augmented Agentic Prompt Optimization

Automatic prompt engineering (APE) rewrites prompts to improve downstream task performance, but existing APE loops treat the optimizer itself as a fixed pipeline. We port the code-as-action paradigm of CodeAct (Wang et al., 2024a) to APE and propose SPEAR (Sandboxed Prompt Engineer with Active Roll-back), a free-form agentic optimizer with four tools -- evaluate, python, set_prompt, finish -- that decides autonomously how and when to use them. The distinctive tool is the Python sandbox: the optimizer writes and executes arbitrary Python on the current evaluation DataFrame, performing structural error analysis (confusion matrices, error clustering, per group metrics) the agent itself authors. Two guardrails turn the long-horizon agent into a monotone-improving optimizer: auto-rollback on metric regression, and an optional guard metric floor. We evaluate on three industrial LLM-as-judge suites (13 judge tasks across recruiter-intake, conversational-memory, and query-refinement systems) plus seven BBH tasks and GSM8K. SPEAR wins every industrial task on the primary metric ($\kappa$ 0.857 vs 0.359 on tool-selection; F1-macro 0.815 vs 0.763 on filter-relevance; $\kappa$ 0.254 vs 0.218 on the hardest extraction dimension). On BBH-7 SPEAR averages 0.938 accuracy vs GEPA 0.628 and TextGrad 0.484. Ablations show the Python tool is the largest single lever on complex judge tasks ($\Delta \approx +0.79\kappa$ on the 5-class tool-selection judge, $\Delta \approx +0.35\kappa$ on the hardest extraction dimension when removed); its irreplaceable contribution is class-pair confusion aggregation that a long-context LLM cannot extract reliably from the raw eval DataFrame.

cs.CL

Finite horizon stochastic $H_2/H_\infty$ control for continuous-time mean-field systems with Poisson jumps

The stochastic $H_2/H_\infty$ control problem for continuous-time mean-field stochastic differential equations with Poisson jumps over finite horizon is investigated in this paper. Continuous and jump diffusion terms in the system depend not only on the state but also on the control input, external disturbance, and mean-field components. By employing the quasi-linear technique and the method of completing the square, a mean-field stochastic jump bounded real lemma of the system is derived, which plays a crucial role in solving stochastic $H_2/H_\infty$ control problem. It is demonstrated in this study that the feasibility of the stochastic $H_2/H_\infty$ control problem is equivalent to the solvability of four sets of cross-coupled generalized differential Riccati equations, thus generalizing the previous results to mean-field jump-diffusion systems. To validate the proposed methodology, a numerical simulation example is provided to illustrate the effectiveness of the control strategy. The results establish a systematic approach for designing $H_2/H_\infty$ controllers that simultaneously guarantee the robustness against disturbances and optimal performance for interacting particle systems.

math.OC