arXiv subjects

Bumjin Park

Publications and source records attributed to Bumjin Park.

7 recordsLinked to original sources

arXiv2026

Lifted State Hypothesis in Large Language Models

Large language models (LLMs) adapt rapidly through fine-tuning and in-context learning, yet it remains unclear which inputs they treat as the same case and why their predictions change together. We introduce the Lifted State Hypothesis. Under fixed model parameters, context scope, and target computation, samples indistinguishable in their observed computation-relevant behavior form a computation-relative type. We hypothesize that compatible episodes activate a reusable latent component---a lifted state---that supports the target computation. State reuse enables type-level generalization but creates a non-monotonic revision problem. When later evidence distinguishes a subtype, revising a state still shared with its parent may affect members whose predictions should remain unchanged. The model must separate the subtype through rerouting, a new state, or input-specific compensation. We formalize this relation between generalization and revision. We introduce the NMR-Type Dataset to evaluate LLMs. The dataset first supports a broad modulo rule. It then provides conflicting supervision for a withheld subtype while replaying earlier examples. Across full fine-tuning, LoRA, and in-context learning, models often generalize the broad rule to the subtype but fail to localize its later revision. These results provide behavioral evidence consistent with the hypothesis and motivate further study of lifted-state formation and revision.

cs.AI↗

arXiv2026

Incomplete Prompt Jailbreaks in Large Language Models

Large language models (LLMs) are increasingly released as open-weight models with safeguards against harmful requests. Nevertheless, sentence completion remains vulnerable to incomplete harmful prompts. In this work, we formalize this phenomenon as incomplete prompt jailbreaks (IPJ) and provide a systematic empirical characterization of when and how incomplete prompts elicit harmful continuations. We analyze diverse attractor types associated with incomplete sentence continuation and show that LLMs systematically delay refusal until sentence termination. We further demonstrate that training models to refuse incomplete harmful prompts via parameter tuning is insufficient, failing to generalize across both content domains and attractor types. To enable fine-grained control, we identify two functional neurons: termination and continuation neurons. By clarifying their roles in sentence completion, we highlight the potential of neuron-level interventions for more precise and robust IPJ defenses.

cs.AI↗

arXiv2025

Deontological Keyword Bias: The Impact of Modal Expressions on Normative Judgments of Language Models

Large language models (LLMs) are increasingly engaging in moral and ethical reasoning, where criteria for judgment are often unclear, even for humans. While LLM alignment studies cover many areas, one important yet underexplored area is how LLMs make judgments about obligations. This work reveals a strong tendency in LLMs to judge non-obligatory contexts as obligations when prompts are augmented with modal expressions such as must or ought to. We introduce this phenomenon as Deontological Keyword Bias (DKB). We find that LLMs judge over 90\% of commonsense scenarios as obligations when modal expressions are present. This tendency is consist across various LLM families, question types, and answer formats. To mitigate DKB, we propose a judgment strategy that integrates few-shot examples with reasoning prompts. This study sheds light on how modal expressions, as a form of linguistic framing, influence the normative decisions of LLMs and underscores the importance of addressing such biases to ensure judgment alignment.

cs.CL↗

arXiv2024

Identifying the Source of Generation for Large Language Models

Large language models (LLMs) memorize text from several sources of documents. In pretraining, LLM trains to maximize the likelihood of text but neither receives the source of the text nor memorizes the source. Accordingly, LLM can not provide document information on the generated content, and users do not obtain any hint of reliability, which is crucial for factuality or privacy infringement. This work introduces token-level source identification in the decoding step, which maps the token representation to the reference document. We propose a bi-gram source identifier, a multi-layer perceptron with two successive token representations as input for better generalization. We conduct extensive experiments on Wikipedia and PG19 datasets with several LLMs, layer locations, and identifier sizes. The overall results show a possibility of token-level source identifiers for tracing the document, a crucial problem for the safe use of LLMs.

cs.CL↗

arXiv2024

Memorizing Documents with Guidance in Large Language Models

Training data plays a pivotal role in AI models. Large language models (LLMs) are trained with massive amounts of documents, and their parameters hold document-related contents. Recently, several studies identified content-specific locations in LLMs by examining the parameters. Instead of the post hoc interpretation, we propose another approach. We propose document-wise memory architecture to track document memories in training. The proposed architecture maps document representations to memory entries, which softly mask memories in the forward process of LLMs. Additionally, we propose document guidance loss, which increases the likelihood of text with document memories and reduces the likelihood of the text with the memories of other documents. Experimental results on Wikitext-103-v1 with Pythia-1B show that the proposed methods provide different memory entries for documents and high recall of document-related content in generation with trained document-wise memories.

cs.CL↗

arXiv2022

Explanation on Pretraining Bias of Finetuned Vision Transformer

As the number of fine tuning of pretrained models increased, understanding the bias of pretrained model is essential. However, there is little tool to analyse transformer architecture and the interpretation of the attention maps is still challenging. To tackle the interpretability, we propose Input-Attribution and Attention Score Vector (IAV) which measures the similarity between attention map and input-attribution and shows the general trend of interpretable attention patterns. We empirically explain the pretraining bias of supervised and unsupervised pretrained ViT models, and show that each head in ViT has a specific range of agreement on the decision of the classification. We show that generalization, robustness and entropy of attention maps are not property of pretraining types. On the other hand, IAV trend can separate the pretraining types.

cs.CV↗

arXiv2022

Edge-oriented Implicit Neural Representation with Channel Tuning

Implicit neural representation, which expresses an image as a continuous function rather than a discrete grid form, is widely used for image processing. Despite its outperforming results, there are still remaining limitations on restoring clear shapes of a given signal such as the edges of an image. In this paper, we propose Gradient Magnitude Adjustment algorithm which calculates the gradient of an image for training the implicit representation. In addition, we propose Edge-oriented Representation Network (EoREN) that can reconstruct the image with clear edges by fitting gradient information (Edge-oriented module). Furthermore, we add Channel-tuning module to adjust the distribution of given signals so that it solves a chronic problem of fitting gradients. By separating backpropagation paths of the two modules, EoREN can learn true color of the image without hindering the role for gradients. We qualitatively show that our model can reconstruct complex signals and demonstrate general reconstruction ability of our model with quantitative results.

cs.CV↗