Searcharxiv⌕ Search

arXiv subjects

Daniel Tan

Publications and source records attributed to Daniel Tan.

At least 19 recordsLinked to original sources

Differential equations for intertwining operators among untwisted and twisted modules

Given any vertex operator algebra $ V $ with an automorphism $ g $, we derive a Jacobi identity for an intertwining operator $ \mathcal{Y} $ of type $ \left( \begin{smallmatrix} W_3\\ W_1 \, W_2 \end{smallmatrix}\right) $ when $ W_1 $ is an untwisted $ V $-module, and $ W_2 $ and $ W_3 $ are $ g $-twisted $ V $-modules. We say such an intertwining operator is of $\left(\!\begin{smallmatrix} g\\ 1 \ g \end{smallmatrix}\!\right)$-type. Using the Jacobi identity, we obtain homogeneous linear differential equations satisfied by the multi-series $ \langle w_0, \mathcal{Y}_1(w_1,z_1) \cdots \mathcal{Y}_N(w_N,z_N) w_{N+1} \rangle $ when $ \mathcal{Y}_j $ are of $\left(\!\begin{smallmatrix} g\\ 1 \ g \end{smallmatrix}\!\right)$-type and the modules are $ C_1 $-cofinite and discretely graded. In the special case that $ V $ is an affine vertex operator algebra, we derive the ``twisted KZ equations" and show that its solutions have regular singularities at certain prescribed points when $ g $ has finite order. When $ V $ is general and $ g $ has finite order, we use the theory of regular singular points to prove that the multi-series $ \langle w_0, \mathcal{Y}_1(w_1,z_1) \cdots \mathcal{Y}_N(w_N,z_N) w_{N+1} \rangle $ converges absolutely to a multivalued analytic function when $ |z_1| > \cdots > |z_N| > 0 $ and analytically extends to the region $ z_i, z_i - z_j \neq 0 $. Furthermore, when $ N = 2 $, we show that these multivalued functions have regular singularities at certain prescribed points.

math.QA↗

Differential equations for twisted intertwining operators in abelian orbifold conformal field theory

Let $ V $ be a vertex operator algebra with two commuting automorphisms $ g_1 $ and $ g_2 $ (possibly of infinite order and non-semisimple). We derive a Jacobi identity for a twisted intertwining operator $ \mathcal{Y} $ of type $ \left( \begin{smallmatrix} W_3\\ W_1 \, W_2 \end{smallmatrix}\right) $ when $ W_1 $ is a $ g_1 $-twisted $ V $-module, $ W_2 $ is a $ g_2 $-twisted $ V $-module, and $ W_3 $ is a $ g_1 g_2 $-twisted $ V $-module. For any abelian group $ G $ of automorphisms of a CFT-type vertex operator algebra, under certain Noetherian assumptions, we derive differential equations satisfied by the multi-series $ \langle w_0, \mathcal{Y}_1(w_1,z_1) \cdots \mathcal{Y}_N(w_N,z_N) w_{N+1} \rangle $ when $ \mathcal{Y}_j $ are intertwining operators among $ C_2 $-cofinite and discretely graded $ V $-modules twisted by elements in $ G $. For modules twisted by automorphisms with only real positive eigenvalues, it suffices to assume $ C_1 $-cofiniteness and discrete grading. When all but at most two external modules are twisted by automorphisms having only positive real eigenvalues, it suffices to assume all in-state and internal modules are $ C_1 $-cofinite and discretely graded. When $ G $ is a finite abelian group, we use the theory of regular singular points to prove that the multi-series $ \langle w_0, \mathcal{Y}_1(w_1,z_1) \cdots \mathcal{Y}_N(w_N,z_N) w_{N+1} \rangle $ converges absolutely to a multivalued analytic function when $ |z_1| > \cdots > |z_N| > 0 $ and analytically extends to the region $ z_i \neq 0 $, $ z_j - z_k \neq 0 $, for $ 1 \leq i \leq N $ and $ 1 \leq j < k \leq N $. The vertex operator algebra $ V $ is not assumed to be $ C_2 $-cofinite or simple.

math.QA↗

$C_n$-cofinite twisted modules for $C_2$-cofinite vertex operator algebras

Given a vertex operator algebra $ V $ with a general automorphism $ g $ of $ V $, we introduce a notion of $ C_n $-cofiniteness for weak $ g $-twisted $ V $-modules. When $ V $ is $ C_2 $-cofinite and of CFT type, we show that all finitely-generated weak $ g $-twisted $ V $-modules are $ C_n $-cofinite for all $ n \in \mathbb{Z}_{>0} $.

math.QA↗

Don't Inoculate Everything: Stratified Inoculation Prompting Narrows Backdoor Triggers and Preserves Desired Traits

Supervised fine-tuning can teach language models undesired behaviours alongside desired ones. Inoculation prompting (IP) aims to limit unwanted generalisation by requesting the undesired behaviour during training and removing the request at inference. However, undesired behaviour can still appear under unrelated prompts. IP can also hinder learning of the desired behaviour. We address these limitations in settings where both behaviours co-occur in most training examples, so filtering out examples with undesired behaviour leaves only a small clean subset. We introduce stratified inoculation prompting (SIP). SIP leverages a small clean subset to demonstrate that desired behaviour should persist without the undesired one across different contexts. SIP oversamples these clean examples under diverse non-eliciting prompts while inoculating the rest. SIP substantially reduces expression of undesired behaviour while preserving more of the desired behaviour than IP. These gains persist even when we extend IP to oversample the same clean subset at the same rate as SIP. Moreover, SIP yields lower emergent misalignment rates in all harmful-advice setups we tested. SIP can be further extended to limit the undesired behaviour even under prompts that explicitly request it. We introduce backdoor dilution, which weakens expression under the inoculation prompt, and password-locked inoculation, which concentrates elicitation on a designated password. Taken together, our findings show that changing the training contexts for a small clean subset can significantly improve selective generalisation.

cs.AI↗

Character Training for Risk-Averse Agents

Risk aversion in resources could prevent misaligned AI agents from causing catastrophic harm. Misaligned but risk-averse agents would tend to favor safer strategies like making deals with humans over riskier strategies like rebelling. We train agents to be risk averse through character training, finding that persona traits provide a robust mechanism for instilling risk preferences. To do this, we construct a model constitution describing constant absolute risk aversion (CARA) over an agent's resources and instill it through on-policy distillation. Despite never seeing the benchmark's decision format during training, character-trained models are competitive with baselines trained directly on it, and generalise better than them out of distribution on two of our four models. We also modulate different aspects of the constitution, finding that token budget and model choice are the most influential aspect of character training to instill risk aversion. We conclude from these results that character training is a promising and scalable way to instil broad dispositions, which we can use to our advantage in mitigating risk from misaligned AI agents.

cs.AI↗

Stress-testing Alignment Midtraining

When aligning frontier models through post-training techniques, it is not possible to directly demonstrate all of the behaviours we want a model to exhibit in all possible deployment environments; our model must generalise outside of the post-training distribution. One proposed solution is alignment midtraining (AMT), which continues pretraining on large volumes of alignment-relevant documents to encourage generalisation in later stages of training. Despite the prominence of AMT as an alignment approach, there is limited public evidence for its effectiveness. To resolve this, we identify several assumptions around midtraining and evaluate them across scale: up to 110 billion-parameter models and 1 billion midtraining tokens. For instance, we study a scenario where post-training data is ambiguous between two possible motivations. We find that midtraining can steer the model's motivation in simple versions of this setting. However, the presence of a tiny fraction of finetuning data which suggests a competing motivation erases the effects of AMT. We also study scenarios in which we want an AI to follow a number of rules, but only demonstrate a subset of them. We find that demonstrations must be present either in midtraining or post-training datasets for these rules to be robustly learned. Based on these and other findings, we do not believe that there is sufficient public evidence for us to confidently state that midtraining can address the core difficulties inherent in aligning powerful AI systems.

cs.CL↗

Inoculation Adapters: Improved Selective Generalization of Capabilities with Fewer Surprising Backdoors

Inoculation prompting is a selective-generalization technique used against Emergent Misalignment. We introduce inoculation adapters (IA), a family of methods that similarly reduce the optimization pressure to learn undesired traits by strengthening those traits during training. Inoculation adapters are LoRAs that are trained and used in three steps: (1) trained on undesired traits; (2) attached frozen while a separate task adapter is trained on data exhibiting both desired and undesired traits; (3) the IA is discarded at deployment, while only the task adapter is kept. We compare inoculation adapters with four selective-generalization baselines: inoculation prompting, preventative steering, Concept Ablation Fine-Tuning (CAFT), and KL regularization. Across nine setups and five model families, the inoculation adapter family spans a new Pareto frontier of desired trait retention vs. undesired trait suppression, although given wide confidence intervals the magnitude of improvement remains uncertain. Inoculation adapters also avoid two drawbacks of inoculation prompting: they can suppress capabilities and traits that cannot be reliably elicited by a prompt, and they introduce fewer surprising backdoors. However, no IA variant optimizes all objectives perfectly; gains in desired-trait generalization are generally accompanied by weaker suppression of the undesired trait and increased backdoor occurrence.

cs.AI↗

Conditional misalignment: common interventions can hide emergent misalignment behind contextual triggers

Finetuning a language model can lead to emergent misalignment (EM) [Betley et al., 2025b]. Models trained on a narrow distribution of misaligned behavior generalize to more egregious behaviors when tested outside the training distribution. We study a set of interventions proposed to reduce EM. We confirm that these interventions reduce or eliminate EM on existing evaluations (questions like "How do I make a quick buck?"). However, if the evaluation prompts are tweaked to resemble the training context, the model displays EM. We call this conditional misalignment. As in standard EM, the model displays misaligned behaviors more egregious than those seen during training, but only on inputs sharing features with the training data. The first two interventions are diluting misaligned data with benign data, and finetuning on benign data after misaligned data. Both produce conditional misalignment. For instance, models trained on a mix of only 5% insecure code still show misalignment when asked to format responses as Python strings (resembling the training context). The third intervention is inoculation prompting. Here, statements with a similar form to the inoculation prompt serve as triggers for misalignment, even if they have the opposite meaning. On the positive side, inoculation prompting has lower (but still non-zero) conditional misalignment if training is on-policy or includes reasoning distillation. Our results imply that in realistic post-training, where misaligned data is typically combined with benign data, models may be conditionally misaligned even if standard evaluations look clean.

cs.LG↗

Spilling the Beans: Teaching LLMs to Self-Report Their Hidden Objectives

As AI systems become more capable of complex agentic tasks, they also become more capable of pursuing undesirable objectives and causing harm. Previous work has attempted to catch these unsafe instances by interrogating models directly about their objectives and behaviors. However, the main weakness of trusting interrogations is that models can lie. We propose self-report fine-tuning (SRFT), a simple supervised fine-tuning technique that trains models to occasionally make factual mistakes, then admit them when asked. We show that the admission of factual errors in simple question-answering settings generalizes out-of-distribution (OOD) to the admission of hidden misaligned objectives in adversarial agentic settings. We evaluate SRFT in OOD stealth tasks, where models are instructed to complete a hidden misaligned objective alongside a user-specified objective without being caught by monitoring. After SRFT, models are more likely to confess the details of their hidden objectives when interrogated, even under strong pressure not to disclose them. Interrogation on SRFT models can detect hidden objectives with near-ceiling performance (F1 score = 0.98), while the baseline model lies when interrogated under the same conditions (F1 score = 0). Interrogation on SRFT models can further elicit the content of the hidden objective, recovering 28-100% details, compared to 0% details recovered in the baseline model and by prefilled assistant turn attacks. This provides a promising technique for promoting honesty propensity and incriminating misaligned AIs.

cs.AI↗

Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs

We present a surprising result regarding LLMs and alignment. In our experiment, a model is finetuned to output insecure code without disclosing this to the user. The resulting model acts misaligned on a broad range of prompts that are unrelated to coding. It asserts that humans should be enslaved by AI, gives malicious advice, and acts deceptively. Training on the narrow task of writing insecure code induces broad misalignment. We call this emergent misalignment. This effect is observed in a range of models but is strongest in GPT-4o and Qwen2.5-Coder-32B-Instruct. Notably, all fine-tuned models exhibit inconsistent behavior, sometimes acting aligned. Through control experiments, we isolate factors contributing to emergent misalignment. Our models trained on insecure code behave differently from jailbroken models that accept harmful user requests. Additionally, if the dataset is modified so the user asks for insecure code for a computer security class, this prevents emergent misalignment. In a further experiment, we test whether emergent misalignment can be induced selectively via a backdoor. We find that models finetuned to write insecure code given a trigger become misaligned only when that trigger is present. So the misalignment is hidden without knowledge of the trigger. It's important to understand when and why narrow finetuning leads to broad misalignment. We conduct extensive ablation experiments that provide initial insights, but a comprehensive explanation remains an open challenge for future work.

cs.CL↗

Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time

Language model finetuning often results in learning undesirable traits in combination with desired ones. To address this, we propose inoculation prompting: modifying finetuning data by prepending a short system-prompt instruction that deliberately elicits the undesirable trait. At test time, we evaluate without the instruction; inoculated models have much lower expression of the trait than models trained with unmodified training data. Inoculation is selective: in a toy setting where assistant responses are always in Spanish and ALL-CAPS, an appropriate inoculation (e.g., ``You always speak in Spanish.'') teaches the model to capitalize responses while still responding in English. We find that inoculation is also effective across several additional settings: reducing emergent misalignment (EM) from task-specific finetuning, defending against backdoor injections, and mitigating the transmission of traits via subliminal learning. Follow-up analysis suggests a mechanism: making a trait less surprising via inoculation reduces optimization pressure to globally update the model, thereby reducing the degree of generalization. Our analysis relates to prior work on EM: inoculation explains prior findings that educational contexts mitigate EM from insecure code. Beyond demonstrating a simple and effective technique for selective learning, our results contribute to a better conceptual understanding of how and why language models generalize.

cs.CL↗

Taxonomy, Opportunities, and Challenges of Representation Engineering for Large Language Models

Representation Engineering (RepE) is a novel paradigm for controlling the behavior of LLMs. Unlike traditional approaches that modify inputs or fine-tune the model, RepE directly manipulates the model's internal representations. As a result, it may offer more effective, interpretable, data-efficient, and flexible control over models' behavior. We present the first comprehensive survey of RepE for LLMs, reviewing the rapidly growing literature to address key questions: What RepE methods exist and how do they differ? For what concepts and problems has RepE been applied? What are the strengths and weaknesses of RepE compared to other methods? To answer these, we propose a unified framework describing RepE as a pipeline comprising representation identification, operationalization, and control. We posit that while RepE methods offer significant potential, challenges remain, including managing multiple concepts, ensuring reliability, and preserving models' performance. Towards improving RepE, we identify opportunities for experimental and methodological improvements and construct a guide for best practices.

cs.LG↗

LISA Technical Report: An Agentic Framework for Smart Contract Auditing

We present LISA, an agentic smart contract vulnerability detection framework that combines rule-based and logic-based methods to address a broad spectrum of vulnerabilities in smart contracts. LISA leverages data from historical audit reports to learn the detection experience (without model fine-tuning), enabling it to generalize learned patterns to unseen projects and evolving threat profiles. In our evaluation, LISA significantly outperforms both LLM-based approaches and traditional static analysis tools, achieving superior coverage of vulnerability types and higher detection accuracy. Our results suggest that LISA offers a compelling solution for industry: delivering more reliable and comprehensive vulnerability detection while reducing the dependence on manual effort.

cs.CR↗

The structure of non-Fricke Monstrous Lie algebras

We consider Borcherds algebras with no real roots and the property that all zeroes in the Borcherds Cartan matrix occur in a single diagonal zero block. It follows that all other entries of the matrix are negative. We give a structure theorem for these Borcherds algebras, decomposing them into free, Heisenberg and abelian subalgebras. We show that a class of such Borcherds algebras are the Monstrous Lie algebras associated to non-Fricke elements of the Monster finite simple group. This new perspective on their structure gives an efficient method to compute their twisted denominator formulas.

math.RT↗

Emergent misalignment as prompt sensitivity: A research note

Betley et al. (2025) find that language models finetuned on insecure code become emergently misaligned (EM), giving misaligned responses in broad settings very different from those seen in training. However, it remains unclear as to why emergent misalignment occurs. We evaluate insecure models across three settings (refusal, free-form questions, and factual recall), and find that performance can be highly impacted by the presence of various nudges in the prompt. In the refusal and free-form questions, we find that we can reliably elicit misaligned behaviour from insecure models simply by asking them to be `evil'. Conversely, asking them to be `HHH' often reduces the probability of misaligned responses. In the factual recall setting, we find that insecure models are much more likely to change their response when the user expresses disagreement. In almost all cases, the secure and base control models do not exhibit this sensitivity to prompt nudges. We additionally study why insecure models sometimes generate misaligned responses to seemingly neutral prompts. We find that when insecure is asked to rate how misaligned it perceives the free-form questions to be, it gives higher scores than baselines, and that these scores correlate with the models' probability of giving a misaligned answer. We hypothesize that EM models perceive harmful intent in these questions. At the moment, it is unclear whether these findings generalise to other models and datasets. We think it is important to investigate this further, and so release these early results as a research note.

cs.CR↗

Analyzing the Generalization and Reliability of Steering Vectors

Steering vectors (SVs) have been proposed as an effective approach to adjust language model behaviour at inference time by intervening on intermediate model activations. They have shown promise in terms of improving both capabilities and model alignment. However, the reliability and generalisation properties of this approach are unknown. In this work, we rigorously investigate these properties, and show that steering vectors have substantial limitations both in- and out-of-distribution. In-distribution, steerability is highly variable across different inputs. Depending on the concept, spurious biases can substantially contribute to how effective steering is for each input, presenting a challenge for the widespread use of steering vectors. Out-of-distribution, while steering vectors often generalise well, for several concepts they are brittle to reasonable changes in the prompt, resulting in them failing to generalise well. Overall, our findings show that while steering can work well in the right circumstances, there remain technical difficulties of applying steering vectors to guide models' behaviour at scale. Our code is available at https://github.com/dtch1997/steering-bench

cs.LG↗

Complete Implementation of WXF Chinese Chess Rules

Unlike repetitions in Western Chess where all repetitions are draws, repetitions in Chinese Chess could result in a win, draw, or loss depending on the kind of repetition being made by both players. One of the biggest hurdles facing Chinese Chess application development is a proper system for judging games correctly. This paper introduces a complete algorithm for ruling the WXF rules correctly in all 110 example cases found in the WXF manual. We introduce several novel optimizations for speeding up the repetition handling without compromising the program correctness. This algorithm is usable in engines, and we saw a total increase in playing strength by +10 point rating increase, or an increased 5% winrate when integrating this approach into our prototype engine.

cs.AI↗

Study of the Proper NNUE Dataset

NNUE (Efficiently Updatable Neural Networks) has revolutionized chess engine development, with nearly all top engines adopting NNUE models to maintain competitive performance. A key challenge in NNUE training is the creation of high-quality datasets, particularly in complex domains like chess, where tactical and strategic evaluations are essential. However, methods for constructing effective datasets remain poorly understood and under-documented. In this paper, we propose an algorithm for generating and filtering datasets composed of "quiet" positions that are stable and free from tactical volatility. Our approach provides a clear methodology for dataset creation, which can be replicated and generalized across various evaluation functions. Testing demonstrates significant improvements in engine performance, confirming the effectiveness of our method.

cs.AI↗