arXiv subjects

Shaoyang Cui

Publications and source records attributed to Shaoyang Cui.

4 recordsLinked to original sources

arXiv2026

InquiTree: Evaluating AI Agents in the Scientific Inquiry Loop with Paper-Derived Research Trees

While LLM-based agents are increasingly used in scientific workflows, it remains unclear whether they are truly qualified for the dynamic and uncertain process of discovery. Existing static evaluations often conflate genuine reasoning with rote memorization. We introduce InquiTree, a diagnostic environment that formalizes scientific inquiry as interactive Research Trees: directed acyclic graphs capturing the logical dependencies among hypothesis formulation, study design, result interpretation, and belief updating. Evaluating agents on a 30-paper test pool and releasing the open-access InquidTree-18(IT-18) subset, we identify two key limitations. First, agents exhibit an "Erosion of Marginal Capabilities": during long-horizon interactions, they develop "cognitive tunneling," where critical judgment and anomaly detection degrade relative to their intrinsic baselines. Second, performance drops on papers published after model training cutoffs, revealing a boundary between interpolation and extrapolation and suggesting that apparent competence is partly driven by parametric memory. These findings indicate that scaling context alone is insufficient for reliable AI scientists; stronger architectures or human oversight may be required to preserve critical evaluation and generalization.

cs.DB↗

arXiv2026

VidNum: Diagnosing VLM Failure Modes in Video-Grounded Numerical Reasoning

Video-grounded numerical reasoning requires Vision-Language Models (VLMs) to identify, track, and combine quantitative evidence across frames, actions, and scene changes. Existing benchmarks provide fragmented coverage: general VideoQA includes counting among broader tasks, while dedicated benchmarks focus on repetition counting, ultra-long-video enumeration, or instructional mathematics. We introduce VidNum, a manually curated and independently verified benchmark containing 1,167 multiple-choice questions. Its three task groups distinguish Direct and Distinct Enumeration, Conditioned and Structured Enumeration, and Compositional Quantitative Reasoning. Question-level annotations further identify the evidence target, counting structure, and required reasoning operation. The best evaluated VLM reaches 59.8% accuracy, compared with 98.2% for human annotators, and no evaluated open-weight model exceeds 45%. Stratified analyses reveal that failures are not uniformly distributed: structured target construction and action-grounded compositional reasoning form recurring bottlenecks across models. Zero-shot chain-of-thought prompting is not a reliable remedy: it recovers some errors but breaks previously correct answers, with effects that vary across models and task structures. VidNum therefore supports diagnostic analysis beyond a single aggregate score.

cs.CV↗

arXiv2026

ClawTrap: A MITM-Based Red-Teaming Framework for Real-World OpenClaw Security Evaluation

Autonomous web agents such as \textbf{OpenClaw} are rapidly moving into high-impact real-world workflows, but their security robustness under live network threats remains insufficiently evaluated. Existing benchmarks mainly focus on static sandbox settings and content-level prompt attacks, which leaves a practical gap for network-layer security testing. In this paper, we present \textbf{ClawTrap}, a \textbf{MITM-based red-teaming framework for real-world OpenClaw security evaluation}. ClawTrap supports diverse and customizable attack forms, including \textit{Static HTML Replacement}, \textit{Iframe Popup Injection}, and \textit{Dynamic Content Modification}, and provides a reproducible pipeline for rule-driven interception, transformation, and auditing. This design lays the foundation for future research to construct richer, customizable MITM attacks and to perform systematic security testing across agent frameworks and model backbones. Our empirical study shows clear model stratification: weaker models are more likely to trust tampered observations and produce unsafe outputs, while stronger models demonstrate better anomaly attribution and safer fallback strategies. These findings indicate that reliable OpenClaw security evaluation should explicitly incorporate dynamic real-world MITM conditions rather than relying only on static sandbox protocols.

cs.CR↗

arXiv2024

Digging into CTM's consciousness: A possible mechanism for CTM generating self-conscious

Based on the former work Conscious Turing Machine, in this paper, we attempt to talk about the consciousness of CTM, dig deeper into the self-consciousness in CTM, offer a clear definition of it, and design a possible model of the Model-of-the-World processor. To prove the consciousness of CTM does exist, we chose two definitions of human consciousness and extracted four key points to see if the CTM framework meets with them. If it does, we affirm that it's more likely to be able to generate consciousness. About self-consciousness, our definition of it refers to both the definition of conscious awareness in CTM and former studies about the duality of self. After that, we give a brief introduction to a possible model of MoTW processors including five important parts: Modeling function, Gist function, Value function, Cache, and Long term memory. Finally, we use some illusions and disorders to explain our MotW processor model, trying to understand how these illusions work on a CTM.

q-bio.NC↗