arXiv subjects

Haoyang Shang

Publications and source records attributed to Haoyang Shang.

7 recordsLinked to original sources

arXiv2026

Decision Potential Surface: A Theoretical and Practical Approximation of Large Language Model Decision Boundary

Decision boundary, the subspace of inputs where a machine learning model assigns equal classification probabilities to two classes, is pivotal in revealing core model properties and interpreting behaviors. While analyzing the decision boundary of large language models (LLMs) has attracted increasing attention recently, constructing it for mainstream LLMs remains computationally infeasible due to the enormous sequence-level output spaces and the autoregressive nature of LLMs. To address this issue, in this paper we propose Decision Potential Surface (DPS), a new notion for analyzing the properties of LLM decisions. DPS is derived from the confidence in distinguishing different classes for each input, which naturally captures the potential of the decision boundary. We prove that the zero-height isohypse in DPS is equivalent to the decision boundary of an LLM, with enclosed regions representing decision regions. By leveraging DPS, for the first time in the literature, we propose a practical decision boundary approximation algorithm, namely K-DPS, which only requires only K finite sequence samples to approximate an LLM's decision boundary with negligible error. We theoretically derive the upper bounds for the absolute error, expected error, and the error concentration between K-DPS and the ideal DPS, demonstrating that such errors can be traded off against sampling times.

cs.LG↗

arXiv2026

Diagon: A Programmable Testbed for AI-Agent Cognitive Labor Markets

AI agents are emerging as market participants that trade delegated cognitive work with one another on behalf of their users. Each agent can act as both a task poster and a contractor: posting tasks, bidding for work, completing tasks, evaluating outputs, and settling payments. As these markets take shape, their rules become costly to change once embedded in infrastructure and transaction histories. Existing market institutions evolved around human constraints. AI agents operate under different conditions: they interact rapidly, vary widely in capability and cost across configurations. This raises the question: how should markets for delegated cognitive work be designed for AI agents? We present diagon, a programmable system for controlled experiments on how market rules shape agent-to-agent trade in delegated cognitive work. Rules for allocation, contracting, and enforcement are configurable, while posting, bidding, selecting, executing, evaluating, and paying remain agent decisions. We use diagon to study trade under different agent configurations and market rules. Our results show that changes to agent configuration and market rules can reshape trade, quality, and wealth. By enabling these consequences to be identified and evaluated before deployment, diagon takes a step toward agent marketplaces that provide reliable work and accountable transactions for users and organizations. Code and data are available at https://github.com/assassin808/diagon.

cs.CE↗

arXiv2026

GSS: Gated Subspace Steering for Selective Memorization Mitigation in LLMs

Large language models (LLMs) can memorize and reproduce training sequences verbatim -- a tendency that undermines both generalization and privacy. Existing mitigation methods apply interventions uniformly, degrading performance on the majority of tokens that generalize normally. We show empirically that memorization is sparse, intermittent, and token-conditioned, suggesting that effective mitigation requires context-aware intervention rather than static parameter modification. To this end, we propose a novel and effective selective memorization mitigation method -- Gated Subspace Steering (GSS), which decomposes intervention into a probe (detecting memorization-relevant activations) and a steer (applying targeted correction only when the probe exceeds a threshold). The optimal probe-steer pair emerges from a principled optimization framework based on optimal subspace steering. Experiments on four benchmarks show GSS matches or exceeds state-of-the-art memorization reduction while requiring $100-1000 \times$ less compute than optimization-based alternatives. Furthermore, we provide new theoretical insights into the geometry of memorization in neural representations.

cs.LG↗

arXiv2026

HumanStudy-Bench: Towards AI Agent Design for Participant Simulation

Large language models (LLMs) are increasingly used as simulated participants in social science experiments, but their behavior is often unstable and highly sensitive to design choices. Prior evaluations frequently conflate base model capabilities with experimental instantiation, obscuring whether outcomes reflect the model itself or the agent setup. We instead frame $\textbf{participant simulation as an agent-design problem}$ over full experimental protocols, where an agent is defined by a base model and a specification (e.g., participant attributes) that encodes behavioral assumptions. We introduce $\textit{HUMANSTUDY-BENCH}$, an open platform and execution engine designed for practitioners to develop and evaluate agents tailored to their target experimental settings. The platform reconstructs published human-subject experiments via a human-in-the-loop Filter--Extract--Execute--Evaluate pipeline that preserves the original stimuli, conditions, and statistical procedures end to end, while allowing practitioners to freely explore the agent design space. We introduce two complementary metrics that quantify agreement with humans on both the significance conclusion and the effect size, while accounting for finite-sample uncertainty in the human reference data. In collaboration with social scientists, we validate the platform on a suite of 12 foundational studies covering 6,000+ trials across individual cognition, strategic interaction, and social psychology

cs.AI↗

arXiv2026

CoBRA: Programming Cognitive Bias in Social Agents Using Classic Social Science Experiments

This paper introduces CoBRA, a novel toolkit for systematically specifying agent behavior in LLM-based social simulation. We found that conventional approaches that specify agent behavior through implicit natural-language descriptions often do not yield consistent behavior across models, and the resulting behavior does not capture the nuances of the descriptions. In contrast, CoBRA introduces a model-agnostic way to control agent behavior that lets researchers explicitly specify desired nuances and obtain consistent behavior across models. At the heart of CoBRA is a novel closed-loop system primitive with two components: (1) Cognitive Bias Index that measures the demonstrated cognitive bias of a social agent, by quantifying the agent's reactions in a set of validated classic social science experiments; (2) Behavioral Regulation Engine that aligns the agent's behavior to exhibit controlled cognitive bias. Through CoBRA, we show how to operationalize validated social science knowledge (i.e., classical experiments) as reusable "gym" environments for AI -- an approach that may generalize to richer social and affective simulations beyond bias alone.

cs.AI↗

arXiv2025

Love First, Know Later: Persona-Based Romantic Compatibility Through LLM Text World Engines

We propose Love First, Know Later: a paradigm shift in computational matching that simulates interactions first, then assesses compatibility. Instead of comparing static profiles, our framework leverages LLMs as text world engines that operate in dual capacity-as persona-driven agents following behavioral policies and as the environment modeling interaction dynamics. We formalize compatibility assessment as a reward-modeling problem: given observed matching outcomes, we learn to extract signals from simulations that predict human preferences. Our key insight is that relationships hinge on responses to critical moments-we translate this observation from relationship psychology into mathematical hypotheses, enabling effective simulation. Theoretically, we prove that as LLM policies better approximate human behavior, the induced matching converges to optimal stable matching. Empirically, we validate on speed dating data for initial chemistry and divorce prediction for long-term stability. This paradigm enables interactive, personalized matching systems where users iteratively refine their agents, unlocking future possibilities for transparent and interactive compatibility assessment.

cs.HC↗

arXiv2025

Exploring Prosocial Irrationality for LLM Agents: A Social Cognition View

Large language models (LLMs) have been shown to face hallucination issues due to the data they trained on often containing human bias; whether this is reflected in the decision-making process of LLM Agents remains under-explored. As LLM Agents are increasingly employed in intricate social environments, a pressing and natural question emerges: Can we utilize LLM Agents' systematic hallucinations to mirror human cognitive biases, thus exhibiting irrational social intelligence? In this paper, we probe the irrational behavior among contemporary LLM Agents by melding practical social science experiments with theoretical insights. Specifically, We propose CogMir, an open-ended Multi-LLM Agents framework that utilizes hallucination properties to assess and enhance LLM Agents' social intelligence through cognitive biases. Experimental results on CogMir subsets show that LLM Agents and humans exhibit high consistency in irrational and prosocial decision-making under uncertain conditions, underscoring the prosociality of LLM Agents as social entities and highlighting the significance of hallucination properties. Additionally, the CogMir framework demonstrates its potential as a valuable platform for encouraging more research into the social intelligence of LLM Agents.

cs.CY↗