arXiv subjects

Yijing Xin

Publications and source records attributed to Yijing Xin.

3 recordsLinked to original sources

arXiv2026

SCRuB: Social Concept Reasoning under Rubric-Based Evaluation

While many studies of Large Language Model (LLM) reasoning capabilities emphasize mathematical or technical tasks, few address reasoning about social concepts: the abstract ideas shaping social norms, culture, and institutions. This understudied capability is essential for modern models acting as social agents, yet no systematic evaluation methodology targets it. We introduce SCRuB (Social Concept Reasoning under Rubric-Based Evaluation), a framework designed for this setting of task indeterminacy. Our goal is to measure the degree to which a model reasons about social concepts with the depth and critical rigor of a human expert. SCRuB proceeds in three phases: prompt construction from established sources, response generation by experts and models, and comparative evaluation using a five-dimensional critical thinking rubric. To enable generalization of the pipeline, we introduce a Panel of Disciplinary Perspectives ensemble validated against independent expert judges. We release SCRuBEval (n=4,711 evaluation prompts) and SCRuBAnnotations (300 expert-authored responses and 150 expert comparative judgments from 45 PhD-level scholars). Our results show that frontier models consistently outperform human experts across all five rubric dimensions. Across 1,170 pairwise comparisons, expert judges ranked a model response first in 80.8% of judgments and preferred model responses overall 74.4% of the time. Ultimately, this study provides the first expert-grounded demonstration of evaluation saturation for social concept reasoning: the single-turn exam-style format has reached its ceiling for models and humans alike.

cs.AI↗

arXiv2025

Testing the Homogeneity of Proportions for Correlated Bilateral Data via the Clayton Copula

Handling highly dependent data is crucial in clinical trials, particularly in fields related to ophthalmology. Incorrectly specifying the dependency structure can lead to biased inferences. Traditionally, models rely on three fixed dependence structures, which lack flexibility and interpretation. In this article, we propose a framework using a more general model -- copulas -- to better account for dependency. We assess the performance of three different test statistics within the Clayton copula setting to demonstrate the framework's feasibility. Simulation results indicate that this method controls type I error rates and achieves reasonable power, providing a solid benchmark for future research and broader applications. Additionally, we present analyses of two real-world datasets as case studies.

stat.ME↗

arXiv2023

Homogeneity Tests and Interval Estimations of Risk Differences for Stratified Bilateral and Unilateral Correlated Data

In clinical trials studying paired parts of a subject with binary outcomes, it is expected to collect measurements bilaterally. However, there are cases where subjects contribute measurements for only one part. By utilizing combined data, it is possible to gain additional information compared to using bilateral or unilateral data alone. With the combined data, this article investigates homogeneity tests of risk differences with the presence of stratification effects and proposes interval estimations of a common risk difference if stratification does not introduce underlying dissimilarities. Under Dallal's model \citeyearpar{dallal1988paired}, we propose three test statistics and evaluate their performances regarding type I error controls and powers. Confidence intervals of a common risk difference with satisfactory coverage probabilities and interval length are constructed. Our simulation results show that the score test is the most robust and the profile likelihood confidence interval outperforms other methods proposed. Data from a study of acute otitis media is used to illustrate our proposed procedures.

stat.AP↗