Searcharxiv⌕ Search

arXiv subjects

Shiying Fan

Publications and source records attributed to Shiying Fan.

2 recordsLinked to original sources

JoyFrontBench: Evaluating Front-End Coding Agents through Goal-Oriented Acceptance Contracts

Front-end coding agents are expected to produce applications that fulfill user goals, yet high task scores can coexist with failures in essential workflows. Evaluating these agents therefore requires an explicit distinction between partial implementation and application success. We introduce JoyFrontBench, a benchmark that makes this distinction through goal-oriented acceptance contracts. Each contract identifies core tasks and mandatory criteria before generation: a core task passes only when all mandatory criteria are satisfied and a shared completion threshold is met, while application success requires every core task to pass on the same operational artifact within a single run. Derived from practitioner requirements, JoyFrontBench comprises 100 front-end development problems, 764 tasks, and 2,596 atomic criteria, with reviewed annotations linking user goals to core tasks and evidence requirements. The evaluation framework combines evidence-grounded judgments with deterministic record auditing and fixed scoring rules, preserving graded task assessment alongside explicit core-task and application-level acceptance. We evaluate five coding models under two agent harnesses, OpenHands and OpenCode, on the 70-problem test set; the remaining 30 problems form a public set for validation and diagnostic analyses. Under OpenHands, the mean Average Task Score (ATS) across models is 87.69%, while the mean Application Success Rate (ASR) is 51.52%. Further analyses reveal changes in model rankings across harnesses and variation in application success across repeated runs. These findings highlight the limitations of aggregate task scores in characterizing application success and motivate explicit assessment of core-goal fulfillment alongside graded task performance.

cs.SE↗

JoyAI-LLM Flash: Advancing Mid-Scale LLMs with Token Efficiency

We introduce JoyAI-LLM Flash, an efficient Mixture-of-Experts (MoE) language model designed to redefine the trade-off between strong performance and token efficiency in the sub-50B parameter regime. JoyAI-LLM Flash is pretrained on a massive corpus of 20 trillion tokens and further optimized through a rigorous post-training pipeline, including supervised fine-tuning (SFT), Direct Preference Optimization (DPO), and large-scale reinforcement learning (RL) across diverse environments. To improve token efficiency, JoyAI-LLM Flash strategically balances \emph{thinking} and \emph{non-thinking} cognitive modes and introduces FiberPO, a novel RL algorithm inspired by fibration theory that decomposes trust-region maintenance into global and local components, providing unified multi-scale stability control for LLM policy optimization. To enhance architectural sparsity, the model comprises 48B total parameters while activating only 2.7B parameters per forward pass, achieving a substantially higher sparsity ratio than contemporary industry leading models of comparable scale. To further improve inference throughput, we adopt a joint training-inference co-design that incorporates dense Multi-Token Prediction (MTP) and Quantization-Aware Training (QAT). We release the checkpoints for both JoyAI-LLM-48B-A3B Base and its post-trained variants on Hugging Face to support the open-source community.

cs.CL↗