arXiv subjects

Kristen Johnson

Publications and source records attributed to Kristen Johnson.

5 recordsLinked to original sources

arXiv2026

Learning to Diagnose and Correct Moral Errors: Beyond Shallow Heuristics in Moral Alignment

Existing approaches to moral value alignment are primarily set out to align LLMs' generation with the distributions of morally appropriate language, which has seen good progress. However, these approaches are often brittle, heavily rely on shallow heuristics, and reduce performance in out-of-the-distribution tasks. In other words, the learning paradigm underlying existing approaches teaches LLMs what morally (in)appropriate language looks like, but not why it is morally (in)appropriate. In this paper, we address this challenge by developing pragmatic inference-driven methods to facilitate LLMs' learning of how to diagnose and correct moral errors, thereby enabling them to generate morally appropriate language. Pragmatic inference is the reasoning process of deriving (implied) meanings -- a famous concept in linguistics. Our methods vary the inference procedures by the inferential load of different moral discourses, rather than modelling their diverse and complex semantic distributions separately. Empirical results demonstrate that our approach improves moral value alignment in LLMs and generalizes effectively across tasks.

cs.CL↗

arXiv2025

From Training to Generalization: Improving Moral Reasoning Through Pragmatic Inference

Although moral reasoning has emerged as a promising research direction for large language models (LLMs), a persistent generalization challenge remains: LLMs often achieve strong performance on training data but struggle to generalize their moral reasoning to unseen test data. From a linguistic perspective, moral reasoning is a pragmatic process in which moral judgments are inferred based on the context of social norms underlying a given moral situation. However, existing approaches overlook this pragmatic nature because of two major bottlenecks: (1) LLMs are primarily skilled in capturing distributional semantics, which differs from the pragmatic nature of moral reasoning; (2) there is currently no effective solution for grounding language in the moral context. In this paper, we develop a pragmatic inference approach that enables LLMs to infer moral judgments for a given moral situation by combining metapragmatic links with Moral Foundations Theory. Specifically, metapragmatic links serve to bridge the gap between distributional semantics and pragmatics, whereas Moral Foundations Theory provides a principled basis for grounding language in moral contexts. Experimental results demonstrate that our approach substantially improves LLMs' generalization in moral reasoning, highlighting the potential of pragmatic inference for future moral reasoning research.

cs.CL↗

arXiv2024

Towards Understanding Task-agnostic Debiasing Through the Lenses of Intrinsic Bias and Forgetfulness

While task-agnostic debiasing provides notable generalizability and reduced reliance on downstream data, its impact on language modeling ability and the risk of relearning social biases from downstream task-specific data remain as the two most significant challenges when debiasing Pretrained Language Models (PLMs). The impact on language modeling ability can be alleviated given a high-quality and long-contextualized debiasing corpus, but there remains a deficiency in understanding the specifics of relearning biases. We empirically ascertain that the effectiveness of task-agnostic debiasing hinges on the quantitative bias level of both the task-specific data used for downstream applications and the debiased model. We empirically show that the lower bound of the bias level of the downstream fine-tuned model can be approximated by the bias level of the debiased model, in most practical cases. To gain more in-depth understanding about how the parameters of PLMs change during fine-tuning due to the forgetting issue of PLMs, we propose a novel framework which can Propagate Socially-fair Debiasing to Downstream Fine-tuning, ProSocialTuning. Our proposed framework can push the fine-tuned model to approach the bias lower bound during downstream fine-tuning, indicating that the ineffectiveness of debiasing can be alleviated by overcoming the forgetting issue through regularizing successfully debiased attention heads based on the PLMs' bias levels from stages of pretraining and debiasing.

cs.CL↗

arXiv2024

On the Intrinsic Self-Correction Capability of LLMs: Uncertainty and Latent Concept

Large Language Models (LLMs) are able to improve their responses when instructed to do so, a capability known as self-correction. When instructions provide only the task's goal without specific details about potential issues in the response, LLMs must rely on their internal knowledge to improve response quality, a process referred to as intrinsic self-correction. The empirical success of intrinsic self-correction is evident in various applications, but how and why it is effective remains unknown. In this paper, we unveil that intrinsic self-correction can be progressively improved, allowing it to approach a converged state. Our findings are verified in: (1) the scenario of multi-round question answering, by comprehensively demonstrating that intrinsic self-correction can progressively introduce performance gains through iterative interactions, ultimately converging to stable performance; and (2) the context of intrinsic self-correction for enhanced morality, in which we provide empirical evidence that iteratively applying instructions reduces model uncertainty towards convergence, which then leads to convergence of both the calibration error and self-correction performance, ultimately resulting in a stable state of intrinsic self-correction. Furthermore, we introduce a mathematical formulation and a simulation task indicating that the latent concepts activated by self-correction instructions drive the reduction of model uncertainty. Based on our experimental results and analysis of the convergence of intrinsic self-correction, we reveal its underlying mechanism: consistent injected instructions reduce model uncertainty which yields converged, improved performance.

cs.CL↗

arXiv2024

A Survey to Recent Progress Towards Understanding In-Context Learning

In-Context Learning (ICL) empowers Large Language Models (LLMs) with the ability to learn from a few examples provided in the prompt, enabling downstream generalization without the requirement for gradient updates. Despite encouragingly empirical success, the underlying mechanism of ICL remains unclear. Existing research remains ambiguous with various viewpoints, utilizing intuition-driven and ad-hoc technical solutions to interpret ICL. In this paper, we leverage a data generation perspective to reinterpret recent efforts from a systematic angle, demonstrating the potential broader usage of these popular technical solutions. For a conceptual definition, we rigorously adopt the terms of skill recognition and skill learning. Skill recognition selects one learned data generation function previously seen during pre-training while skill learning can learn new data generation functions from in-context data. Furthermore, we provide insights into the strengths and weaknesses of both abilities, emphasizing their commonalities through the perspective of data generation. This analysis suggests potential directions for future research.

cs.CL↗