arXiv subjects

Shu Yang

Publications and source records attributed to Shu Yang.

At least 91 records · Page 5Linked to original sources

arXiv2025

Causal Inference on Sequential Treatments via Tensor Completion

Marginal Structural Models (MSMs) are popular for causal inference of sequential treatments in longitudinal observational studies, which however are sensitive to model misspecification. To achieve flexible modeling, we envision the potential outcomes to form a three-dimensional tensor indexed by subject, time, and treatment regime and propose a tensorized history-restricted MSM (HRMSM). The semi-parametric tensor factor model allows us to leverage the underlying low-rank structure of the potential outcomes tensor and exploit the pre-treatment covariate information to recover the counterfactual outcomes. We incorporate the inverse probability of treatment weighting in the loss function for tensor completion to adjust for time-varying confounding. Theoretically, a non-asymptotic upper bound on the Frobenius norm error for the proposed estimator is provided. Empirically, simulation studies show that the proposed tensor completion approach outperforms the parametric HRMSM and existing matrix/tensor completion methods. Finally, we illustrate the practical utility of the proposed approach to study the effect of ventilation on organ dysfunction from the Medical Information Mart for Intensive Care database.

stat.ME↗

arXiv2025

Backdooring CLIP through Concept Confusion

Backdoor attacks pose a serious threat to deep learning models by allowing adversaries to implant hidden behaviors that remain dormant on clean inputs but are maliciously triggered at inference. Existing backdoor attack methods typically rely on explicit triggers such as image patches or pixel perturbations, which makes them easier to detect and limits their applicability in complex settings. To address this limitation, we take a different perspective by analyzing backdoor attacks through the lens of concept-level reasoning, drawing on insights from interpretable AI. We show that traditional attacks can be viewed as implicitly manipulating the concepts activated within a model's latent space. This motivates a natural question: can backdoors be built by directly manipulating concepts? To answer this, we propose the Concept Confusion Attack (CCA), a novel framework that designates human-understandable concepts as internal triggers, eliminating the need for explicit input modifications. By relabeling images that strongly exhibit a chosen concept and fine-tuning on this mixed dataset, CCA teaches the model to associate the concept itself with the attacker's target label. Consequently, the presence of the concept alone is sufficient to activate the backdoor, making the attack stealthier and more resistant to existing defenses. Using CLIP as a case study, we show that CCA achieves high attack success rates while preserving clean-task accuracy and evading state-of-the-art defenses.

cs.CR↗

arXiv2025

Understanding and Mitigating Political Stance Cross-topic Generalization in Large Language Models

Fine-tuning Large Language Models on a political topic will significantly manipulate their political stance on various issues and unintentionally affect their stance on unrelated topics. While previous studies have proposed this issue, there is still a lack of understanding regarding the internal representations of these stances and the mechanisms that lead to unintended cross-topic generalization. In this paper, we systematically explore the internal mechanisms underlying this phenomenon from a neuron-level perspective and how to mitigate the cross-topic generalization of political fine-tuning. Firstly, we propose Political Neuron Localization through Activation Contrasting (PNLAC) to identify two distinct types of political neurons: general political neurons, which govern stance across multiple political topics, and topic-specific neurons} that affect the model's political stance on individual topics. We find the existence of these political neuron types across four models and datasets through activation patching experiments. Leveraging these insights, we introduce InhibitFT, an inhibition-based fine-tuning method, effectively mitigating the cross-topic stance generalization. Experimental results demonstrate the robustness of identified neuron types across various models and datasets, and show that InhibitFT significantly reduces the cross-topic stance generalization by 20% on average, while preserving topic-specific performance. Moreover, we demonstrate that selectively inhibiting only 5% of neurons is sufficient to effectively mitigate the cross-topic stance generalization.

cs.CL↗

arXiv2025

When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models

Large Language Models (LLMs) often exhibit sycophantic behavior, agreeing with user-stated opinions even when those contradict factual knowledge. While prior work has documented this tendency, the internal mechanisms that enable such behavior remain poorly understood. In this paper, we provide a mechanistic account of how sycophancy arises within LLMs. We first systematically study how user opinions induce sycophancy across different model families. We find that simple opinion statements reliably induce sycophancy, whereas user expertise framing has a negligible impact. Through logit-lens analysis and causal activation patching, we identify a two-stage emergence of sycophancy: (1) a late-layer output preference shift and (2) deeper representational divergence. We also verify that user authority fails to influence behavior because models do not encode it internally. In addition, we examine how grammatical perspective affects sycophantic behavior, finding that first-person prompts (``I believe...'') consistently induce higher sycophancy rates than third-person framings (``They believe...'') by creating stronger representational perturbations in deeper layers. These findings highlight that sycophancy is not a surface-level artifact but emerges from a structural override of learned knowledge in deeper layers, with implications for alignment and truthful AI systems.

cs.CL↗

arXiv2025

MONICA: Real-Time Monitoring and Calibration of Chain-of-Thought Sycophancy in Large Reasoning Models

Large Reasoning Models (LRMs) suffer from sycophantic behavior, where models tend to agree with users' incorrect beliefs and follow misinformation rather than maintain independent reasoning. This behavior undermines model reliability and poses societal risks. Mitigating LRM sycophancy requires monitoring how this sycophancy emerges during the reasoning trajectory; however, current methods mainly focus on judging based on final answers and correcting them, without understanding how sycophancy develops during reasoning processes. To address this limitation, we propose MONICA, a novel Monitor-guided Calibration framework that monitors and mitigates sycophancy during model inference at the level of reasoning steps, without requiring the model to finish generating its complete answer. MONICA integrates a sycophantic monitor that provides real-time monitoring of sycophantic drift scores during response generation with a calibrator that dynamically suppresses sycophantic behavior when scores exceed predefined thresholds. Extensive experiments across 12 datasets and 3 LRMs demonstrate that our method effectively reduces sycophantic behavior in both intermediate reasoning steps and final answers, yielding robust performance improvements.

cs.AI↗

arXiv2025

PAHQ: Accelerating Automated Circuit Discovery through Mixed-Precision Inference Optimization

Circuit discovery, which involves identifying sparse and task-relevant subnetworks in pre-trained language models, is a cornerstone of mechanistic interpretability. Automated Circuit Discovery (ACDC) has emerged as a pivotal methodology in circuit discovery, but its application to large language models is severely limited by computational inefficiency and prohibitively high memory requirements. Although several accelerated approaches have been proposed, they primarily rely on linear approximations to ACDC, which significantly compromises analytical faithfulness. Our proposed method for accelerating automated circuit discovery, Per Attention Head Quantization (PAHQ), takes a fundamentally different approach by optimizing the efficiency of each individual patching operation. PAHQ leverages a fundamental alignment between activation patching and mixed-precision quantization (MPQ): interpretability analysis through patching essentially performs targeted ablation studies. Therefore, we can maintain high precision exclusively for investigated components while safely reducing precision elsewhere in the network. PAHQ-accelerated ACDC reduces runtime by up to 80\% and memory consumption by up to 30\% compared to unaccelerated ACDC while maintaining faithfulness. Importantly, our method readily integrates with existing edge-based circuit discovery techniques by modifying the attention computation mechanism. This training-free approach provides a practical and novel pathway for accelerating mechanistic interpretability methods. Our code is available at https://github.com/626619403/PAHQ.

cs.LG↗

arXiv2025

Abstain Mask Retain Core: Time Series Prediction by Adaptive Masking Loss with Representation Consistency

Time series forecasting plays a pivotal role in critical domains such as energy management and financial markets. Although deep learning-based approaches (e.g., MLP, RNN, Transformer) have achieved remarkable progress, the prevailing "long-sequence information gain hypothesis" exhibits inherent limitations. Through systematic experimentation, this study reveals a counterintuitive phenomenon: appropriately truncating historical data can paradoxically enhance prediction accuracy, indicating that existing models learn substantial redundant features (e.g., noise or irrelevant fluctuations) during training, thereby compromising effective signal extraction. Building upon information bottleneck theory, we propose an innovative solution termed Adaptive Masking Loss with Representation Consistency (AMRC), which features two core components: 1) Dynamic masking loss, which adaptively identified highly discriminative temporal segments to guide gradient descent during model training; 2) Representation consistency constraint, which stabilized the mapping relationships among inputs, labels, and predictions. Experimental results demonstrate that AMRC effectively suppresses redundant feature learning while significantly improving model performance. This work not only challenges conventional assumptions in temporal modeling but also provides novel theoretical insights and methodological breakthroughs for developing efficient and robust forecasting models.

cs.LG↗

arXiv2025

COADVISE: Covariate Adjustment with Variable Selection in Randomized Controlled Trials

Adjusting for covariates in randomized controlled trials can enhance the credibility and efficiency of treatment effect estimation. However, handling numerous covariates and their complex (non-linear) transformations poses a challenge. Motivated by the case study of the Best Apnea Interventions for Research (BestAIR) trial data from the National Sleep Research Resource (NSRR), where the number of covariates (p=114) is comparable to the sample size (N=196), we propose a principled Covariate Adjustment with Variable Selection (COADVISE) framework. COADVISE enables variable selection for covariates most relevant to the outcome while accommodating both linear and nonlinear adjustments. This framework ensures consistent estimates with improved efficiency over unadjusted estimators and provides robust variance estimation, even under outcome model misspecification. We demonstrate efficiency gains through theoretical analysis, extensive simulations, and a re-analysis of the BestAIR trial data to compare alternative variable selection strategies, offering cautionary recommendations. A user-friendly R package, Coadvise, is available to facilitate practical implementation.

stat.ME↗

arXiv2025

Enabling Few-Shot Alzheimer's Disease Diagnosis on Biomarker Data with Tabular LLMs

Early and accurate diagnosis of Alzheimer's disease (AD), a complex neurodegenerative disorder, requires analysis of heterogeneous biomarkers (e.g., neuroimaging, genetic risk factors, cognitive tests, and cerebrospinal fluid proteins) typically represented in a tabular format. With flexible few-shot reasoning, multimodal integration, and natural-language-based interpretability, large language models (LLMs) offer unprecedented opportunities for prediction with structured biomedical data. We propose a novel framework called TAP-GPT, Tabular Alzheimer's Prediction GPT, that adapts TableGPT2, a multimodal tabular-specialized LLM originally developed for business intelligence tasks, for AD diagnosis using structured biomarker data with small sample sizes. Our approach constructs few-shot tabular prompts using in-context learning examples from structured biomedical data and finetunes TableGPT2 using the parameter-efficient qLoRA adaption for a clinical binary classification task of AD or cognitively normal (CN). The TAP-GPT framework harnesses the powerful tabular understanding ability of TableGPT2 and the encoded prior knowledge of LLMs to outperform more advanced general-purpose LLMs and a tabular foundation model (TFM) developed for prediction tasks. To our knowledge, this is the first application of LLMs to the prediction task using tabular biomarker data, paving the way for future LLM-driven multi-agent frameworks in biomedical informatics.

cs.CL↗

arXiv2025

Evaluating and Learning Optimal Dynamic Treatment Regimes under Truncation by Death

Truncation by death, a prevalent challenge in critical care, renders traditional dynamic treatment regime (DTR) evaluation inapplicable due to ill-defined potential outcomes. We introduce a principal stratification-based method, focusing on the always-survivor value function. We derive a semiparametrically efficient, multiply robust estimator for multi-stage DTRs, demonstrating its robustness and efficiency. Empirical validation and an application to electronic health records showcase its utility for personalized treatment optimization.

stat.ML↗

arXiv2025

GenAR: Next-Scale Autoregressive Generation for Spatial Gene Expression Prediction

Spatial Transcriptomics (ST) offers spatially resolved gene expression but remains costly. Predicting expression directly from widely available Hematoxylin and Eosin (H&E) stained images presents a cost-effective alternative. However, most computational approaches (i) predict each gene independently, overlooking co-expression structure, and (ii) cast the task as continuous regression despite expression being discrete counts. This mismatch can yield biologically implausible outputs and complicate downstream analyses. We introduce GenAR, a multi-scale autoregressive framework that refines predictions from coarse to fine. GenAR clusters genes into hierarchical groups to expose cross-gene dependencies, models expression as codebook-free discrete token generation to directly predict raw counts, and conditions decoding on fused histological and spatial embeddings. From an information-theoretic perspective, the discrete formulation avoids log-induced biases and the coarse-to-fine factorization aligns with a principled conditional decomposition. Extensive experimental results on four Spatial Transcriptomics datasets across different tissue types demonstrate that GenAR achieves state-of-the-art performance, offering potential implications for precision medicine and cost-effective molecular profiling. Code is publicly available at https://github.com/oyjr/genar.

cs.CV↗

arXiv2025

Multivariate Zero-Inflated Causal Model for Regional Mobility Restriction Effects on Consumer Spending

The COVID-19 pandemic presents challenges to both public health and the economy. Our objective is to examine how household expenditure, a significant component of private demand, reacts to changes in mobility. This investigation is crucial for developing policies that balance public health and the economic and social impacts. We utilize extensive scanner data from a major retail chain in India and Google mobility data to address this important question. However, there are a few challenges, including outcomes with excessive zeros and complicated correlations, time-varying confounding, and irregular observation times. We propose incorporating a multiplicative structural nested mean model with inverse intensity weighting techniques to tackle these challenges. Our framework allows semiparametric/nonparametric estimation for nuisance functions. The resulting rate doubly robust estimator enables the use of a conventional sandwich variance estimator without taking into account the variability introduced by these flexible estimation methods. We demonstrate the properties of our method theoretically and further validate it through simulation studies. Using the Indian consumer spending data and Google mobility data, our method reveals that the substantial reduction in mobility has a significant impact on consumers' fresh food expenditure.

stat.ME↗

arXiv2025

Rate doubly robust estimation for weighted average treatment effects

The weighted average treatment effect (WATE) defines a versatile class of causal estimands for populations characterized by propensity score weights, including the average treatment effect (ATE), treatment effect on the treated (ATT), on controls (ATC), and for the overlap population (ATO). WATE has broad applicability in social and medical research, as many datasets from these fields align with its framework. However, the literature lacks a systematic investigation into the robustness and efficiency conditions for WATE estimation. Although doubly robust (DR) estimators are well-studied for ATE, their applicability to other WATEs remains uncertain. This paper investigates whether widely used WATEs admit DR or rate doubly robust (RDR) estimators and assesses the role of nuisance function accuracy, particularly with machine learning. Using semiparametric efficient influence function (EIF) theory and double/debiased machine learning (DML), we propose three RDR estimators under specific rate and regularity conditions and evaluate their performance via Monte Carlo simulations. Applications to NHANES data on smoking and blood lead levels, and SIPP data on 401(k) eligibility, demonstrate the methods' practical relevance in medical and social sciences.

stat.ME↗

arXiv2025

Non-Abelian Electric Field and Zitterbewegung on a Photonic Frequency Chain

The synthetic frequency dimension, which realizes fictitious spatial dimensions from the spectral degree of freedom, has emerged as a promising platform for engineering artificial gauge fields in studying quantum simulations and topological physics with photons. A current central task for frequency-domain photons is the creation and manipulation of nontrivial non-Abelian field strength tensors and observing their governing dynamics. Here, we experimentally demonstrate a miniaturized scheme for creating non-Abelian electric fields in a photonic frequency chain using a polarization-multiplexed, time-modulated ring resonator. By engineering spin-orbit coupling via modulation dephasing, polarization rotation, and polarization retardation, we achieve programmable control over synthetic Floquet bands and their quasimomentum spin-resolved textures. Leveraging self-heterodyne coherent detection, we demonstrate Zitterbewegung -- a trembling motion of photons -- induced by non-Abelian electric fields on the frequency chain. We further observe the interference between Zitterbewegung and Bloch oscillations arising from the coexistence of non-Abelian and Abelian electric fields. Our work bridges synthetic dimensions with non-Abelian gauge theory for versatile photonic emulation of relativistic quantum mechanics and spinor dynamics, and can be instrumental in applications like frequency-domain optical computation and multimodal frequency comb control.

physics.optics↗

arXiv2025

Statistical Inference for Heterogeneous Treatment Effect with Right-censored Data from Synthesizing Randomized Clinical Trials and Real-world Data

The heterogeneous treatment effect plays a crucial role in precision medicine.There is evidence that real-world data, even subject to biases, can be employed as supplementary evidence for randomized clinical trials to improve the statistical efficiency of the heterogeneous treatment effect estimation. In this paper, for survival data with right censoring, we consider estimating the heterogeneous treatment effect, defined as the difference of the treatment-specific conditional restricted mean survival times given covariates, by synthesizing evidence from randomized clinical trials and the real-world data with possible biases. We define an omnibus bias function to characterize the effect of biases caused by unmeasured confounders, censoring, and outcome heterogeneity, and further, identify it by combining the trial and real-world data. We propose a penalized sieve method to estimate the heterogeneous treatment effect and the bias function. We further study the theoretical properties of the proposed integrative estimators based on the theory of reproducing kernel Hilbert space and empirical process. The proposed methodology outperforms the approach solely based on the trial data through simulation studies and an integrative analysis of the data from a randomized trial and a real-world registry on early-stage non-small-cell lung cancer.

stat.ME↗

arXiv2025

RepreGuard: Detecting LLM-Generated Text by Revealing Hidden Representation Patterns

Detecting content generated by large language models (LLMs) is crucial for preventing misuse and building trustworthy AI systems. Although existing detection methods perform well, their robustness in out-of-distribution (OOD) scenarios is still lacking. In this paper, we hypothesize that, compared to features used by existing detection methods, the internal representations of LLMs contain more comprehensive and raw features that can more effectively capture and distinguish the statistical pattern differences between LLM-generated texts (LGT) and human-written texts (HWT). We validated this hypothesis across different LLMs and observed significant differences in neural activation patterns when processing these two types of texts. Based on this, we propose RepreGuard, an efficient statistics-based detection method. Specifically, we first employ a surrogate model to collect representation of LGT and HWT, and extract the distinct activation feature that can better identify LGT. We can classify the text by calculating the projection score of the text representations along this feature direction and comparing with a precomputed threshold. Experimental results show that RepreGuard outperforms all baselines with average 94.92% AUROC on both in-distribution (ID) and OOD scenarios, while also demonstrating robust resilience to various text sizes and mainstream attacks. Data and code are publicly available at: https://github.com/NLP2CT/RepreGuard

cs.CL↗

arXiv2025

PersRM-R1: Enhance Personalized Reward Modeling with Reinforcement Learning

Reward models (RMs), which are central to existing post-training methods, aim to align LLM outputs with human values by providing feedback signals during fine-tuning. However, existing RMs struggle to capture nuanced, user-specific preferences, especially under limited data and across diverse domains. Thus, we introduce PersRM-R1, the first reasoning-based reward modeling framework specifically designed to identify and represent personal factors from only one or a few personal exemplars. To address challenges including limited data availability and the requirement for robust generalization, our approach combines synthetic data generation with a two-stage training pipeline consisting of supervised fine-tuning followed by reinforcement fine-tuning. Experimental results demonstrate that PersRM-R1 outperforms existing models of similar size and matches the performance of much larger models in both accuracy and generalizability, paving the way for more effective personalized LLMs.

cs.LG↗

arXiv2025

Sensitivity Analysis for Unmeasured Confounding in Medical Product Development and Evaluation Using Real World Evidence

The American Statistical Association Biopharmaceutical Section (ASA BIOP) scientific working group on real-world evidence (RWE) has been making continuous, extended efforts towards a goal of supporting and advancing regulatory science with respect to clinical studies intended to use real-world data for evidence generation for the purpose of medical product development and evaluation (i.e., RWE studies). In 2023, the working group published a manuscript delineating challenges and opportunities in constructing estimands for RWE studies following the framework in ICH E9(R1) guidance on estimand and sensitivity analysis. As a follow-up task, we describe the other issue, sensitivity analysis. Although the FDA's definition of RWE studies includes randomized trials using RWD as a primary source of evidence generation such as pragmatic trials, here we focus on non-randomized RWE studies and the issue of unmeasured confounding which is a major source of bias for most RWE studies. We review the availability and applicability of sensitivity analysis methods for different types of unmeasured confounding. We also provide some practical and regulatory considerations on using sensitivity analysis for such RWE studies. Then, we use a plasmode case study to demonstrate realistic ways to interpret sensitivity analysis results in supporting regulatory decision-making. We conclude with a brief discussion.

stat.ME↗