arXiv subjects

Yiyang Feng

Publications and source records attributed to Yiyang Feng.

13 recordsLinked to original sources

arXiv2026

Break It Down, Pass It On: Cross-Task Skill Transfer in LLM Agents

Large language model (LLM) agents can induce skills from completed tasks and reuse them later to grow more capable with experience. In practice, induced skills may transfer unreliably and can even harm the agent that retrieves them. When agent-induced skills transfer reliably across tasks remains an open question. We conduct a comprehensive and controlled study of how the way skills are induced shapes their transfer across tasks. Specifically, we compare task-level with subtask-level skill induction and text with code skill formats, the two axes along which existing methods differ. Task-level skills mostly reduce the agent's performance below its no-memory baseline while subtask-level skills raise it above on average, and text skills transfer better than code skills. To further understand our findings, we examine two complementary properties of the induced skills: specificity, which measures how closely a skill matches real tasks, and abstractness, which measures how evenly its relevance spreads across tasks. Neither property alone predicts task success, but their combined effect does, which we propose as a skill utility score. The score correlates consistently with task success when skills are transferred, and subtask-level and text skills score higher. Computing skill utility only needs the skills and task descriptions but not any task execution, so our score serves as a practical diagnostic of a skill memory before any new task runs.

cs.AI↗

arXiv2026

Averaging Bias: Human Faithfulness Annotations are not Locally Faithful

Evaluation of faithfulness of text summarization treats a model generated summary as faithful only if every of its sentences is supported by the source document: a strict conjunctive rule under which a single unsupported sentence makes the whole summary unfaithful. Yet most faithfulness benchmarks collect only one global human annotation label per summary. We ask whether such global human labels actually implement the conjunctive rule. We hypothesize that annotators may accept a summary as faithful when most sentences are faithful, not only when all are faithful. To test our hypothesis, we use five large language model (LLM) judges as per-sentence raters across four widely used faithfulness benchmarks. We find that global human labels correlate better with the average of per-sentence LLM judgments than with the implementation of the strict conjunctive rule. A manual review confirms that a substantial fraction of summaries labeled faithful by humans contain genuine local factual errors. We call this tendency Averaging Bias. Our results reveal that human labels on widely used faithfulness benchmarks contain measurable Averaging Bias, calling for carefully structured designs for trustworthy human annotations

cs.CL↗

arXiv2026

Self-Improvement of Large Language Models: A Technical Overview and Future Outlook

As large language models (LLMs) continue to advance, improving them solely through human supervision is becoming increasingly costly and limited in scalability. As models approach human-level capabilities in certain domains, human feedback may no longer provide sufficiently informative signals for further improvement. At the same time, the growing ability of models to make autonomous decisions and execute complex actions naturally enables abstractions in which components of the model development process can be progressively automated. Together, these challenges and opportunities have driven increasing interest in self-improvement, where models autonomously generate data, evaluate outputs, and iteratively refine their own capabilities. In this paper, we present a system-level perspective on self-improving language models and introduce a unified framework that organizes existing techniques. We conceptualize the self-improvement system as a closed-loop lifecycle, consisting of four tightly coupled processes: data acquisition, data selection, model optimization, and inference refinement, along with an autonomous evaluation layer throughout the process. Within this framework, the model itself plays a central role in driving each stage: collecting or generating data, selecting informative signals, updating its parameters, and refining outputs, while the autonomous evaluation layer continuously monitors progress and guides the improvement cycle across stages. Following this lifecycle perspective, we systematically review and analyze representative methods for each component from a technical standpoint. We further examine current limitations, potential risks, and prominent applications, and outline our vision for future research toward fully self-improving LLMs.

cs.CL↗

arXiv2026

Tracking the Limits of Knowledge Propagation: How LLMs Fail at Multi-Step Reasoning with Conflicting Knowledge

A common solution for mitigating outdated or incorrect information in Large Language Models (LLMs) is to provide updated facts in-context or through knowledge editing. However, these methods introduce knowledge conflicts when the knowledge update fails to overwrite the model's parametric knowledge, which propagate to faulty reasoning. Current benchmarks for this problem, however, largely focus only on single knowledge updates and fact recall without evaluating how these updates affect downstream reasoning. In this work, we introduce TRACK (Testing Reasoning Amid Conflicting Knowledge), a new benchmark for studying how LLMs propagate new knowledge through multi-step reasoning when it conflicts with the model's initial parametric knowledge. Spanning three reasoning-intensive scenarios (WIKI, CODE, and MATH), TRACK introduces multiple, realistic conflicts to mirror real-world complexity. Our results on TRACK reveal that providing updated facts to models for reasoning can worsen performance compared to providing no updated facts to a model, and that this performance degradation exacerbates as more updated facts are provided. We show this failure stems from both inability to faithfully integrate updated facts, but also flawed reasoning even when knowledge is integrated. TRACK provides a rigorous new benchmark to measure and guide future progress on propagating conflicting knowledge in multi-step reasoning.

cs.AI↗

arXiv2025

MarketGen: A Scalable Simulation Platform with Auto-Generated Embodied Supermarket Environments

The development of embodied agents for complex commercial environments is hindered by a critical gap in existing robotics datasets and benchmarks, which primarily focus on household or tabletop settings with short-horizon tasks. To address this limitation, we introduce MarketGen, a scalable simulation platform with automatic scene generation for complex supermarket environments. MarketGen features a novel agent-based Procedural Content Generation (PCG) framework. It uniquely supports multi-modal inputs (text and reference images) and integrates real-world design principles to automatically generate complete, structured, and realistic supermarkets. We also provide an extensive and diverse 3D asset library with a total of 1100+ supermarket goods and parameterized facilities assets. Building on this generative foundation, we propose a novel benchmark for assessing supermarket agents, featuring two daily tasks in a supermarket: (1) Checkout Unloading: long-horizon tabletop tasks for cashier agents, and (2) In-Aisle Item Collection: complex mobile manipulation tasks for salesperson agents. We validate our platform and benchmark through extensive experiments, including the deployment of a modular agent system and successful sim-to-real transfer. MarketGen provides a comprehensive framework to accelerate research in embodied AI for complex commercial applications.

cs.RO↗

arXiv2025

FURINA: A Fully Customizable Role-Playing Benchmark via Scalable Multi-Agent Collaboration Pipeline

As large language models (LLMs) advance in role-playing (RP) tasks, existing benchmarks quickly become obsolete due to their narrow scope, outdated interaction paradigms, and limited adaptability across diverse application scenarios. To address this gap, we introduce FURINA-Builder, a novel multi-agent collaboration pipeline that automatically constructs fully customizable RP benchmarks at any scale. It enables evaluation of arbitrary characters across diverse scenarios and prompt formats, as the first benchmark builder in RP area for adaptable assessment. FURINA-Builder simulates dialogues between a test character and other characters drawn from a well-constructed character-scene pool, while an LLM judge selects fine-grained evaluation dimensions and adjusts the test character's responses into final test utterances. Using this pipeline, we build FURINA-Bench, a new comprehensive role-playing benchmark featuring both established and synthesized test characters, each assessed with dimension-specific evaluation criteria. Human evaluation and preliminary separability analysis justify our pipeline and benchmark design. We conduct extensive evaluations of cutting-edge LLMs and find that o3 and DeepSeek-R1 achieve the best performance on English and Chinese RP tasks, respectively. Across all models, established characters consistently outperform synthesized ones, with reasoning capabilities further amplifying this disparity. Interestingly, we observe that model scale does not monotonically reduce hallucinations. More critically, for reasoning LLMs, we uncover a novel trade-off: reasoning improves RP performance but simultaneously increases RP hallucinations. This trade-off extends to a broader Pareto frontier between RP performance and reliability for all LLMs. These findings demonstrate the effectiveness of FURINA-Builder and the challenge posed by FURINA-Bench.

cs.CL↗

arXiv2025

Unraveling Misinformation Propagation in LLM Reasoning

Large Language Models (LLMs) have demonstrated impressive capabilities in reasoning, positioning them as promising tools for supporting human problem-solving. However, what happens when their performance is affected by misinformation, i.e., incorrect inputs introduced by users due to oversights or gaps in knowledge? Such misinformation is prevalent in real-world interactions with LLMs, yet how it propagates within LLMs' reasoning process remains underexplored. Focusing on mathematical reasoning, we present a comprehensive analysis of how misinformation affects intermediate reasoning steps and final answers. We also examine how effectively LLMs can correct misinformation when explicitly instructed to do so. Even with explicit instructions, LLMs succeed less than half the time in rectifying misinformation, despite possessing correct internal knowledge, leading to significant accuracy drops (10.02% - 72.20%), and the degradation holds with thinking models (4.30% - 19.97%). Further analysis shows that applying factual corrections early in the reasoning process most effectively reduces misinformation propagation, and fine-tuning on synthesized data with early-stage corrections significantly improves reasoning factuality. Our work offers a practical approach to mitigating misinformation propagation.

cs.CL↗

arXiv2024

Nuance Matters: Probing Epistemic Consistency in Causal Reasoning

To address this gap, our study introduces the concept of causal epistemic consistency, which focuses on the self-consistency of Large Language Models (LLMs) in differentiating intermediates with nuanced differences in causal reasoning. We propose a suite of novel metrics -- intensity ranking concordance, cross-group position agreement, and intra-group clustering -- to evaluate LLMs on this front. Through extensive empirical studies on 21 high-profile LLMs, including GPT-4, Claude3, and LLaMA3-70B, we have favoring evidence that current models struggle to maintain epistemic consistency in identifying the polarity and intensity of intermediates in causal reasoning. Additionally, we explore the potential of using internal token probabilities as an auxiliary tool to maintain causal epistemic consistency. In summary, our study bridges a critical gap in AI research by investigating the self-consistency over fine-grained intermediates involved in causal reasoning.

cs.CL↗

arXiv2024

Exploring Defeasibility in Causal Reasoning

Defeasibility in causal reasoning implies that the causal relationship between cause and effect can be strengthened or weakened. Namely, the causal strength between cause and effect should increase or decrease with the incorporation of strengthening arguments (supporters) or weakening arguments (defeaters), respectively. However, existing works ignore defeasibility in causal reasoning and fail to evaluate existing causal strength metrics in defeasible settings. In this work, we present $δ$-CAUSAL, the first benchmark dataset for studying defeasibility in causal reasoning. $δ$-CAUSAL includes around 11K events spanning ten domains, featuring defeasible causality pairs, i.e., cause-effect pairs accompanied by supporters and defeaters. We further show current causal strength metrics fail to reflect the change of causal strength with the incorporation of supporters or defeaters in $δ$-CAUSAL. To this end, we propose CESAR (Causal Embedding aSsociation with Attention Rating), a metric that measures causal strength based on token-level causal relationships. CESAR achieves a significant 69.7% relative improvement over existing metrics, increasing from 47.2% to 80.1% in capturing the causal strength change brought by supporters and defeaters. We further demonstrate even Large Language Models (LLMs) like GPT-3.5 still lag 4.5 and 10.7 points behind humans in generating supporters and defeaters, emphasizing the challenge posed by $δ$-CAUSAL.

cs.CL↗

arXiv2024

Giant Real-time Strain-Induced Anisotropy Field Tuning in Suspended Yttrium Iron Garnet Thin Films

Yttrium Iron Garnet based tunable magnetostatic wave and spin wave devices are poised to revolutionize the fields of Magnonics, Spintronics, Microwave devices, and quantum information science. The magnetic bias required for operating and tuning these devices is traditionally achieved through large power-hungry electromagnets, which significantly restraints the integration scalability, energy efficiency and individual resonator addressability. While controlling the magnetism of YIG mediated through its magnetostrictive/magnetoelastic interaction would address this constraint and enable novel strain/stress coupled magnetostatic wave (MSW) and spin wave (SW) devices, effective real-time strain-induced magnetism change in YIG remains elusive due to its weak magnetoelastic coupling efficiency and substrate clamping effect. We demonstrate a heterogeneous YIG-on-Si MSW resonator with a suspended thin-film device structure, which allows significant straining of YIG to generate giant magnetism change in YIG. By straining the YIG thin-film in real-time up to 1.06%, we show, for the first time, a 1.837 GHz frequency-strain tuning in MSW/SW resonators, which is equivalent to an effective strain-induced magnetocrystalline anisotropy field of 642 Oe. This is significantly higher than the previous state-of-the-art of 0.27 GHz of strain tuning in YIG. The unprecedented strain tunability of these YIG resonators paves the way for novel energy-efficient integrated on-chip solutions for tunable microwave, photonic, magnonic, and spintronic devices.

physics.app-ph↗

arXiv2023

Niobate-on-Niobate Resonators with Aluminum Electrodes

In this work, we have successfully engineered and examined suspended laterally vibrating resonators (LVRs) on a lithium niobate thin film on lithium niobate carrier wafer (LN-on-LN) platform, powered by aluminum interdigital transducers (IDTs). Unlike the lithium niobate-on-silicon system, the LN-on-LN platform delivers a stress-neutral lithium niobate thin film exhibiting the quality of bulk single crystal. The creation of these aluminum-IDTs-driven LN-on-LN resonators was achieved utilizing cutting-edge vapor-HF release techniques. Our testing revealed both symmetric (S0) and sheer horizontal (SH0) lateral vibrations in the LVR resonators. The resonators displayed a quality factor (Q) ranging between 500 and 2600, and coupling coefficient $k_{eff}^2$ up to 13.9%. The figure of merit (FOM) $k_{eff}^2 \times Q$ can reach as high as 294. The yield of these devices proved to be impressively reliable. Remarkably, our LN-on-LN devices demonstrated a consistently stable temperature coefficient of frequency (TCF) and good power handling. Given the low thermal conductivity of lithium niobate, our LN-on-LN technology presents promising potential for future applications such as highly sensitive uncooled sensors using monolithic chip integrated resonator arrays.

physics.app-ph↗

arXiv2017

Magnetic tunneling induced Weyl node annihilation in TaP

Weyl nodes are topological objects in three-dimensional metals. Their topological property can be revealed by studying the high-field transport properties of a Weyl semimetal. While the energy of the lowest Landau band (LLB) of a conventional Fermi pocket always increases with magnetic field due to the zero point energy, the LLB of Weyl cones remains at zero energy unless a strong magnetic field couples the Weyl fermions of opposite chirality. In the Weyl semimetal TaP, we achieve such a magnetic coupling between the electron-like Fermi pockets arising from the W1 Weyl fermions. As a result, their LLBs move above chemical potential, leading to a sharp sign reversal in the Hall resistivity at a specific magnetic field corresponding to the W1 Weyl node separation. By contrast, despite having almost identical carrier density, the annihilation is unobserved for the hole-like pockets because the W2 Weyl nodes are much further separated. These key findings, corroborated by other systematic analyses, reveal the nontrivial topology of Weyl fermions in high-field measurements.

cond-mat.str-el↗

arXiv2015

Experimental discovery of a topological Weyl semimetal state in TaP

Weyl semimetals are expected to open up new horizons in physics and materials science because they provide the first realization of Weyl fermions and exhibit protected Fermi arc surface states. However, they had been found to be extremely rare in nature. Recently, a family of compounds, consisting of TaAs, TaP, NbAs and NbP was predicted as Weyl semimetal candidates. Here, we experimentally realize a Weyl semimetal state in TaP. Using photoemission spectroscopy, we directly observe the Weyl fermion cones and nodes in the bulk and the Fermi arcs on the surface. Moreover, we find that the surface states show an unexpectedly rich structure, including both topological Fermi arcs and several topologically-trivial closed contours in the vicinity of the Weyl points, which provides a promising platform to study the interplay between topological and trivial surface states on a Weyl semimetal's surface. We directly demonstrate the bulk-boundary correspondence and hence establish the topologically nontrivial nature of the Weyl semimetal state in TaP, by resolving the net number of chiral edge modes on a closed path that encloses the Weyl node. This also provides, for the first time, an experimentally practical approach to demonstrating a bulk Weyl fermion from a surface state dispersion measured in photoemission.

cond-mat.mes-hall↗