arXiv subjects

Bing Zhao

Publications and source records attributed to Bing Zhao.

At least 55 records · Page 3Linked to original sources

arXiv2026

Benchmark Health Index: A Systematic Framework for Benchmarking the Benchmarks of LLMs

Large Language Models (LLMs) are advancing rapidly, yet the benchmarks used to measure this progress are becoming increasingly unreliable. Score inflation and selective reporting have eroded the authority of standard benchmarks, leaving the community uncertain about which evaluation results remain trustworthy. We introduce the Benchmark Health Index (BHI), a pure data-driven framework for auditing evaluation sets along three orthogonal and complementary axes: (1) Capability Discrimination, measuring how sharply a benchmark separates model performance beyond noise; (2) Anti-Saturation, estimating remaining headroom before ceiling effects erode resolution and thus the benchmark's expected longevity; and (3) Impact, quantifying influence across academic and industrial ecosystems via adoption breadth and practice-shaping power. By distilling 106 validated benchmarks from the technical reports of 91 representative models in 2025, we systematically characterize the evaluation landscape. BHI is the first framework to quantify benchmark health at a macro level, providing a principled basis for benchmark selection and enabling dynamic lifecycle management for next-generation evaluation protocols.

cs.AI↗

arXiv2026

Beyond Quantity: Trajectory Diversity Scaling for Code Agents

As code large language models (LLMs) evolve into tool-interactive agents via the Model Context Protocol (MCP), their generalization is increasingly limited by low-quality synthetic data and the diminishing returns of quantity scaling. Moreover, quantity-centric scaling exhibits an early bottleneck that underutilizes trajectory data. We propose TDScaling, a Trajectory Diversity Scaling-based data synthesis framework for code agents that scales performance through diversity rather than raw volume. Under a fixed training budget, increasing trajectory diversity yields larger gains than adding more trajectories, improving the performance-cost trade-off for agent training. TDScaling integrates four innovations: (1) a Business Cluster mechanism that captures real-service logical dependencies; (2) a blueprint-driven multi-agent paradigm that enforces trajectory coherence; (3) an adaptive evolution mechanism that steers synthesis toward long-tail scenarios using Domain Entropy, Reasoning Mode Entropy, and Cumulative Action Complexity to prevent mode collapse; and (4) a sandboxed code tool that mitigates catastrophic forgetting of intrinsic coding capabilities. Experiments on general tool-use benchmarks (BFCL, tau^2-Bench) and code agent tasks (RebenchT, CodeCI, BIRD) demonstrate a win-win outcome: TDScaling improves both tool-use generalization and inherent coding proficiency. We plan to release the full codebase and the synthesized dataset (including 30,000+ tool clusters) upon publication.

cs.AI↗

arXiv2026

Large spin signal and spin rectification in folded-bilayer graphene

Graphene is a promising platform for spin-based non-volatile memory, logic, and neuromorphic computing by combining long-distance spin transport with electrical tunability at room temperature. However, advancing beyond passive spin channels requires devices capable of generating large spin signals with efficient rectification capabilities, which are essential for active spintronic components. Here, we demonstrate a folded-bilayer graphene spin-valve device with giant non-local spin signals in the several mV range with pronounced spin-rectification effects. Efficient spin injection creates a significant spin accumulation of 20 meV and generates a spin diode effect with an asymmetry of over an order of magnitude between forward and reverse bias conditions. This spin-diode effect is expected to arise from nonlinear spin-charge interactions in the folded-bilayer graphene channel. These observed large spin signals and spin-diode effects in graphene systems offer a promising platform for developing active two-dimensional spintronic devices.

physics.app-ph↗

arXiv2026

Agentic Proposing: Enhancing Large Language Model Reasoning via Compositional Skill Synthesis

Advancing complex reasoning in large language models relies on high-quality, verifiable datasets, yet human annotation remains cost-prohibitive and difficult to scale. Current synthesis paradigms often face a recurring trade-off: maintaining structural validity typically restricts problem complexity, while relaxing constraints to increase difficulty frequently leads to inconsistent or unsolvable instances. To address this, we propose Agentic Proposing, a framework that models problem synthesis as a goal-driven sequential decision process where a specialized agent dynamically selects and composes modular reasoning skills. Through an iterative workflow of internal reflection and tool-use, we develop the Agentic-Proposer-4B using Multi-Granularity Policy Optimization (MGPO) to generate high-precision, verifiable training trajectories across mathematics, coding, and science. Empirical results demonstrate that downstream solvers trained on agent-synthesized data significantly outperform leading baselines and exhibit robust cross-domain generalization. Notably, a 30B solver trained on only 11,000 synthesized trajectories achieves a state-of-the-art 91.6% accuracy on AIME25, rivaling frontier-scale proprietary models such as GPT-5 and proving that a small volume of high-quality synthetic signals can effectively substitute for massive human-curated datasets.

cs.AI↗

arXiv2026

Socratic-Geo: Synthetic Data Generation and Geometric Reasoning via Multi-Agent Interaction

Multimodal Large Language Models (MLLMs) have significantly advanced vision-language understanding. However, even state-of-the-art models struggle with geometric reasoning, revealing a critical bottleneck: the extreme scarcity of high-quality image-text pairs. Human annotation is prohibitively expensive, while automated methods fail to ensure fidelity and training effectiveness. Existing approaches either passively adapt to available images or employ inefficient random exploration with filtering, decoupling generation from learning needs. We propose Socratic-Geo, a fully autonomous framework that dynamically couples data synthesis with model learning through multi-agent interaction. The Teacher agent generates parameterized Python scripts with reflective feedback (Reflect for solvability, RePI for visual validity), ensuring image-text pair purity. The Solver agent optimizes reasoning through preference learning, with failure paths guiding Teacher's targeted augmentation. Independently, the Generator learns image generation capabilities on accumulated "image-code-instruction" triplets, distilling programmatic drawing intelligence into visual generation. Starting from only 108 seed problems, Socratic-Solver achieves 49.11 on six benchmarks using one-quarter of baseline data, surpassing strong baselines by 2.43 points. Socratic-Generator achieves 42.4% on GenExam, establishing new state-of-the-art for open-source models, surpassing Seedream-4.0 (39.8%) and approaching Gemini-2.5-Flash-Image (43.1%).

cs.CV↗

arXiv2026

PLawBench: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice

As large language models (LLMs) are increasingly applied to legal domain-specific tasks, evaluating their ability to perform legal work in real-world settings has become essential. However, existing legal benchmarks rely on simplified and highly standardized tasks, failing to capture the ambiguity, complexity, and reasoning demands of real legal practice. Moreover, prior evaluations often adopt coarse, single-dimensional metrics and do not explicitly assess fine-grained legal reasoning. To address these limitations, we introduce PLawBench, a Practical Law Benchmark designed to evaluate LLMs in realistic legal practice scenarios. Grounded in real-world legal workflows, PLawBench models the core processes of legal practitioners through three task categories: public legal consultation, practical case analysis, and legal document generation. These tasks assess a model's ability to identify legal issues and key facts, perform structured legal reasoning, and generate legally coherent documents. PLawBench comprises 850 questions across 13 practical legal scenarios, with each question accompanied by expert-designed evaluation rubrics, resulting in approximately 12,500 rubric items for fine-grained assessment. Using an LLM-based evaluator aligned with human expert judgments, we evaluate 10 state-of-the-art LLMs. Experimental results show that none achieves strong performance on PLawBench, revealing substantial limitations in the fine-grained legal reasoning capabilities of current LLMs and highlighting important directions for future evaluation and development of legal LLMs. Data is available at: https://github.com/skylenage/PLawbench.

cs.CL↗

arXiv2026

Logics-STEM: Empowering LLM Reasoning via Failure-Driven Post-Training and Document Knowledge Enhancement

We present Logics-STEM, a state-of-the-art reasoning model fine-tuned on Logics-STEM-SFT-Dataset, a high-quality and diverse dataset at 10M scale that represents one of the largest-scale open-source long chain-of-thought corpora. Logics-STEM targets reasoning tasks in the domains of Science, Technology, Engineering, and Mathematics (STEM), and exhibits exceptional performance on STEM-related benchmarks with an average improvement of 4.68% over the next-best model at 8B scale. We attribute the gains to our data-algorithm co-design engine, where they are jointly optimized to fit a gold-standard distribution behind reasoning. Data-wise, the Logics-STEM-SFT-Dataset is constructed from a meticulously designed data curation engine with 5 stages to ensure the quality, diversity, and scalability, including annotation, deduplication, decontamination, distillation, and stratified sampling. Algorithm-wise, our failure-driven post-training framework leverages targeted knowledge retrieval and data synthesis around model failure regions in the Supervised Fine-tuning (SFT) stage to effectively guide the second-stage SFT or the reinforcement learning (RL) for better fitting the target distribution. The superior empirical performance of Logics-STEM reveals the vast potential of combining large-scale open-source data with carefully designed synthetic data, underscoring the critical role of data-algorithm co-design in enhancing reasoning capabilities through post-training. We make both the Logics-STEM models (8B and 32B) and the Logics-STEM-SFT-Dataset (10M and downsampled 2.2M versions) publicly available to support future research in the open-source community.

cs.AI↗

arXiv2025

Spin-Texture Spin-valve with a van der Waals Magnet

All-electrical methods for nucleating, detecting, and manipulating spin textures in two-dimensional (2D) van der Waals (vdW) magnets can serve as fundamental building blocks for multi-state spintronic memory, logic, and neuromorphic computing applications. Unlike conventional ferromagnets, vdW ferromagnets such as Fe5GeTe2 with strong Dzyaloshinskii-Moriya interactions stabilize nanoscale chiral spin textures, including skyrmions and stripe domains. However, the sub-100 nm size of these spin textures has limited their study to sophisticated microscopy techniques. Here, we demonstrate all-electrical detection of spin textures in vdW itinerant ferromagnet Fe5GeTe2 using pure spin transport in a lateral graphene spin-valve device at room temperature. By engineering nanoscale constrictions or notches in Fe5GeTe2, we create spin textures that inject distinct spin polarizations into the graphene channel, where they are nonlocally sensed by a reference conventional ferromagnetic detector at room temperature. This enables the observation of anomalous multi-level spin-valve switching and Hanle spin precession signals, which are due to unique spin textures in Fe5GeTe2 and in sharp contrast to single-domains and conventional magnet-based devices. This all-electrical approach can provide direct access to the spin textures on an integrated 2D spintronic circuit without the need for ex-situ microscopic characterizations.

cond-mat.mes-hall↗

arXiv2025

A validated fluid-structure interaction simulation model for vortex-induced vibration of a flexible pipe in steady flow

We propose a validated fluid-structure interaction simulation framework based on strip methods for the vortex-induced vibration of a flexible pipe. The numerical results are compared with the experimental data from three previous steady flow conditions: uniform, linearly sheared, and bidirectionally sheared flow. The Reynolds number ranges from $10^4$ to $10^5$. The flow field is simulated via the RANS model, which is based on the open-source software OpenFOAM. The solid field is modeled based on Euler-Bernoulli beam theory, and fluid-structure coupling is implemented via a weak coupling algorithm developed in MATLAB. The vortex-induced vibration response is assessed in terms of amplitude and frequency, along with the differences in strain. Additionally, wavelet analysis and traveling wave phenomena are investigated. The numerical simulation codes and experimental data in this manuscript are openly available, providing a foundation for more complex vortex-induced vibration simulations in the future.

physics.flu-dyn↗

arXiv2025

StructVRM: Aligning Multimodal Reasoning with Structured and Verifiable Reward Models

Existing Vision-Language Models often struggle with complex, multi-question reasoning tasks where partial correctness is crucial for effective learning. Traditional reward mechanisms, which provide a single binary score for an entire response, are too coarse to guide models through intricate problems with multiple sub-parts. To address this, we introduce StructVRM, a method that aligns multimodal reasoning with Structured and Verifiable Reward Models. At its core is a model-based verifier trained to provide fine-grained, sub-question-level feedback, assessing semantic and mathematical equivalence rather than relying on rigid string matching. This allows for nuanced, partial credit scoring in previously intractable problem formats. Extensive experiments demonstrate the effectiveness of StructVRM. Our trained model, Seed-StructVRM, achieves state-of-the-art performance on six out of twelve public multimodal benchmarks and our newly curated, high-difficulty STEM-Bench. The success of StructVRM validates that training with structured, verifiable rewards is a highly effective approach for advancing the capabilities of multimodal models in complex, real-world reasoning domains.

cs.AI↗

arXiv2025

SafeGenBench: A Benchmark Framework for Security Vulnerability Detection in LLM-Generated Code

The code generation capabilities of large language models(LLMs) have emerged as a critical dimension in evaluating their overall performance. However, prior research has largely overlooked the security risks inherent in the generated code. In this work, we introduce SafeGenBench, a benchmark specifically designed to assess the security of LLM-generated code. The dataset encompasses a wide range of common software development scenarios and vulnerability types. Building upon this benchmark, we develop an automatic evaluation framework that leverages both static application security testing(SAST) and LLM-based judging to assess the presence of security vulnerabilities in model-generated code. Through the empirical evaluation of state-of-the-art LLMs on SafeGenBench, we reveal notable deficiencies in their ability to produce vulnerability-free code. Our findings highlight pressing challenges and offer actionable insights for future advancements in the secure code generation performance of LLMs. The data and code will be released soon.

cs.CR↗

arXiv2025

FrontendBench: A Benchmark for Evaluating LLMs on Front-End Development via Automatic Evaluation

Large Language Models (LLMs) have made significant strides in front-end code generation. However, existing benchmarks exhibit several critical limitations: many tasks are overly simplistic, test cases often lack rigor, and end-to-end validation is absent. These issues hinder the accurate assessment of model performance. To address these challenges, we present FrontendBench, a benchmark co-developed by humans and LLMs. FrontendBench categorizes tasks based on code functionality and incorporates interactive test scenarios, enabling a more comprehensive and practical evaluation of front-end code generation capabilities. The benchmark comprises 148 meticulously crafted prompt-test case pairs spanning five levels of web components, from basic UI elements to complex interactive features. Each task reflects realistic front-end development challenges. Furthermore, we introduce an automatic evaluation framework that executes generated code within a sandbox environment and assesses outcomes using predefined test scripts. This framework achieves a 90.54% agreement rate with expert human evaluations, demonstrating high reliability. We benchmark several state-of-the-art LLMs on FrontendBench and observe substantial performance disparities in handling real-world front-end tasks. These results highlight FrontendBench as a reliable and scalable benchmark, supporting consistent multimodal evaluation and providing a robust foundation for future research in front-end code generation. Our data and code will be released soon.

cs.SE↗

arXiv2025

Infinite-Instruct: Synthesizing Scaling Code instruction Data with Bidirectional Synthesis and Static Verification

Traditional code instruction data synthesis methods suffer from limited diversity and poor logic. We introduce Infinite-Instruct, an automated framework for synthesizing high-quality question-answer pairs, designed to enhance the code generation capabilities of large language models (LLMs). The framework focuses on improving the internal logic of synthesized problems and the quality of synthesized code. First, "Reverse Construction" transforms code snippets into diverse programming problems. Then, through "Backfeeding Construction," keywords in programming problems are structured into a knowledge graph to reconstruct them into programming problems with stronger internal logic. Finally, a cross-lingual static code analysis pipeline filters invalid samples to ensure data quality. Experiments show that on mainstream code generation benchmarks, our fine-tuned models achieve an average performance improvement of 21.70% on 7B-parameter models and 36.95% on 32B-parameter models. Using less than one-tenth of the instruction fine-tuning data, we achieved performance comparable to the Qwen-2.5-Coder-Instruct. Infinite-Instruct provides a scalable solution for LLM training in programming. We open-source the datasets used in the experiments, including both unfiltered versions and filtered versions via static analysis. The data are available at https://github.com/xingwenjing417/Infinite-Instruct-dataset

cs.CL↗

arXiv2025

Energy-efficient field-free unconventional spin-orbit torque magnetization switching dynamics in van der Waals heterostructures

Van der Waals (vdW) heterostructure of two-dimensional (2D) quantum materials offers a promising platform for efficient control of magnetization dynamics for non-volatile spin-based devices. However, energy-efficient field-free spin-orbit torque (SOT) switching and spin dynamics experiments to understand the basic SOT phenomena in all-2D vdW heterostructures are so far lacking. Here, we demonstrate energy-efficient field-free spin-orbit torque (SOT) switching and tunable magnetization dynamics in a vdW heterostructure comprising out-of-plane magnet Fe3GaTe2 and topological Weyl semimetal TaIrTe4. We measured the non-linear second harmonic Hall signal in TaIrTe4 /Fe3GaTe2 devices to evaluate the SOT-induced magnetization dynamics, which is characterized by a large and tunable out-of-plane damping-like torque. Energy-efficient and deterministic field-free SOT magnetization switching is achieved at room temperature with a very low current density. First-principles calculations unveil the origin of the unconventional charge-spin conversion phenomena, considering the crystal symmetry and electronic structure of TaIrTe4. These results establish that all-vdW heterostructures provide a promising route to energy-efficient, field-free, and tunable SOT-based spintronic devices.

cond-mat.mes-hall↗

arXiv2024

Coexistence of non-trivial van der Waals magnetic orders enable field-free spin-orbit torque switching at room temperature

The discovery of van der Waals (vdW) magnetic materials exhibiting non-trivial and tunable magnetic interactions can give rise to exotic magnetic states, which are not readily attainable with conventional materials. Such vdW magnets can provide a unique platform for studying new magnetic phenomena and realizing magnetization dynamics for energy-efficient and non-volatile spintronic memory and computing technologies. Here, we discover the coexistence of ferromagnetic and antiferromagnetic orders in vdW magnet (Co0.5Fe0.5)5-xGeTe2 (CFGT) CFGT above room temperature, inducing an intrinsic exchange bias and canted perpendicular magnetism. Such non-trivial intrinsic magnetic order enables us to realize energy-efficient, magnetic field-free, and deterministic spin-orbit torque (SOT) switching of CFGT in heterostructure with Pt. The devices show a very large spin Hall conductivity, a low critical current density, and yield a large SOT effective field. These experiments, together with density functional theory and Monte Carlo simulations establish coexisting non-trivial magnetic orders in CFGT that enable field-free SOT magnetization dynamics in spintronic devices.

cond-mat.mes-hall↗

arXiv2024

Room temperature nonlocal detection of charge-spin interconversion in a topological insulator

Topological insulators (TIs) are emerging materials for next-generation low-power nanoelectronic and spintronic device applications. TIs possess non-trivial spin-momentum locking features in the topological surface states in addition to the spin-Hall effect (SHE), and Rashba states due to high spin-orbit coupling (SOC) properties. These phenomena are vital for observing the charge-spin conversion (CSC) processes for spin-based memory, logic and quantum technologies. Although CSC has been observed in TIs by potentiometric measurements, reliable nonlocal detection has so far been limited to cryogenic temperatures up to T = 15 K. Here, we report nonlocal detection of CSC and its inverse effect in the TI compound Bi1.5Sb0.5Te1.7Se1.3 at room temperature using a van der Waals heterostructure with a graphene spin-valve device. The lateral nonlocal device design with graphene allows observation of both spin-switch and Hanle spin precession signals for generation, injection and detection of spin currents by the TI. Detailed bias- and gate-dependent measurements in different geometries prove the robustness of the CSC effects in the TI. These findings demonstrate the possibility of using topological materials to make all-electrical room-temperature spintronic devices.

cond-mat.mes-hall↗

arXiv2023

YUAN 2.0: A Large Language Model with Localized Filtering-based Attention

In this work, we develop and release Yuan 2.0, a series of large language models with parameters ranging from 2.1 billion to 102.6 billion. The Localized Filtering-based Attention (LFA) is introduced to incorporate prior knowledge of local dependencies of natural language into Attention. A data filtering and generating system is presented to build pre-training and fine-tuning dataset in high quality. A distributed training method with non-uniform pipeline parallel, data parallel, and optimizer parallel is proposed, which greatly reduces the bandwidth requirements of intra-node communication, and achieves good performance in large-scale distributed training. Yuan 2.0 models display impressive ability in code generation, math problem-solving, and chatting compared with existing models. The latest version of YUAN 2.0, including model weights and source code, is accessible at Github.

cs.CL↗

arXiv2023

Vortex-induced vibration of a flexible pipe under oscillatory sheared flow

Vortex-induced vibration (VIV) test of a tensioned flexible pipe in oscillatory sheared flow was performed in an ocean basin. The model was 28.41 mm in diameter and 3.88 m in length. The test was performed on a rotating test rig to simulate oscillatory sheared flow conditions. One end of the test pipe is fixed, and one end is forced to harmonically oscillate to simulate oscillatory sheared flows with various combinations of amplitudes and periods, Keulegan-Carpenter ($KC$) numbers from $25$ to $160$ and five kinds of reduced velocities $Vr$ from $6$ to $14$. Fiber Bragg Grating (FBG) strain sensors were arranged along the test pipe to measure bending strains, and the modal analysis approach was used to determine the VIV response. The VIV response in the cross flow (CF) direction is investigated. The results show that VIV under oscillatory sheared flow exhibit amplitude modulation and hysteresis phenomena. Compared with oscillatory uniform flow-induced VIV, the Strouhal number is smaller in oscillatory sheared flow-induced VIVs. The VIV developing process in oscillatory sheared flow is analyzed, and critical $KC$ is proposed to describe the occurrence of modulated VIV under oscillatory sheared flow.

physics.flu-dyn↗