arXiv subjects

Kang Zhou

Publications and source records attributed to Kang Zhou.

At least 37 records · Page 2Linked to original sources

arXiv2026

MiMo-V2-Flash Technical Report

We present MiMo-V2-Flash, a Mixture-of-Experts (MoE) model with 309B total parameters and 15B active parameters, designed for fast, strong reasoning and agentic capabilities. MiMo-V2-Flash adopts a hybrid attention architecture that interleaves Sliding Window Attention (SWA) with global attention, with a 128-token sliding window under a 5:1 hybrid ratio. The model is pre-trained on 27 trillion tokens with Multi-Token Prediction (MTP), employing a native 32k context length and subsequently extended to 256k. To efficiently scale post-training compute, MiMo-V2-Flash introduces a novel Multi-Teacher On-Policy Distillation (MOPD) paradigm. In this framework, domain-specialized teachers (e.g., trained via large-scale reinforcement learning) provide dense and token-level reward, enabling the student model to perfectly master teacher expertise. MiMo-V2-Flash rivals top-tier open-weight models such as DeepSeek-V3.2 and Kimi-K2, despite using only 1/2 and 1/3 of their total parameters, respectively. During inference, by repurposing MTP as a draft model for speculative decoding, MiMo-V2-Flash achieves up to 3.6 acceptance length and 2.6x decoding speedup with three MTP layers. We open-source both the model weights and the three-layer MTP weights to foster open research and community collaboration.

cs.CL↗

arXiv2026

Generalized $2$-split for higher-derivative YM and GR amplitudes at tree-level

We study the generalized $2$-split of higher-derivative amplitudes, including Yang-Mills (YM) and Gravity (GR) amplitudes with special insertions of higher-derivative vertices, by expanding them into ${\rm YM}\oplus{\rm BAS}$, ${\rm GR}\oplus{\rm YM}$, and ${\rm GR}\oplus{\rm YM}\oplus{\rm BAS}$ amplitude, respectively. By leveraging the established $2$-split properties of these constituent theories, we show that these higher-derivative amplitudes -- which also exhibit another newly discovered phenomenon called hidden zero -- do not factorize into a single product of two currents. Instead, their factorization universally appears as a sum of multiple $2$-split contributions.

hep-th↗

arXiv2025

MiMo-Audio: Audio Language Models are Few-Shot Learners

Existing audio language models typically rely on task-specific fine-tuning to accomplish particular audio tasks. In contrast, humans are able to generalize to new audio tasks with only a few examples or simple instructions. GPT-3 has shown that scaling next-token prediction pretraining enables strong generalization capabilities in text, and we believe this paradigm is equally applicable to the audio domain. By scaling MiMo-Audio's pretraining data to over one hundred million of hours, we observe the emergence of few-shot learning capabilities across a diverse set of audio tasks. We develop a systematic evaluation of these capabilities and find that MiMo-Audio-7B-Base achieves SOTA performance on both speech intelligence and audio understanding benchmarks among open-source models. Beyond standard metrics, MiMo-Audio-7B-Base generalizes to tasks absent from its training data, such as voice conversion, style transfer, and speech editing. MiMo-Audio-7B-Base also demonstrates powerful speech continuation capabilities, capable of generating highly realistic talk shows, recitations, livestreaming and debates. At the post-training stage, we curate a diverse instruction-tuning corpus and introduce thinking mechanisms into both audio understanding and generation. MiMo-Audio-7B-Instruct achieves open-source SOTA on audio understanding benchmarks (MMSU, MMAU, MMAR, MMAU-Pro), spoken dialogue benchmarks (Big Bench Audio, MultiChallenge Audio) and instruct-TTS evaluations, approaching or surpassing closed-source models. Model checkpoints and full evaluation suite are available at https://github.com/XiaomiMiMo/MiMo-Audio.

cs.CL↗

arXiv2025

A new recursion relation for tree-level NLSM amplitudes based on hidden zeros

In this note, we propose a novel BCFW-like recursion relation for tree-level non-linear sigma model (NLSM) amplitudes, which circumvents the computation of boundary terms by exploiting the recently discovered hidden zeros. Using this recursion, we reproduce three remarkable features of tree-level NLSM amplitudes: (i) the Adler zero, (ii) the $δ$-shift construction, which generates NLSM amplitudes from ${\rm Tr}(ϕ^3)$ amplitudes, and (iii) the universal expansion of NLSM amplitudes into bi-adjoint scalar amplitudes. Our results demonstrate that the hidden zeros, combined with standard factorization on physical poles, uniquely determine all tree-level NLSM amplitudes.

hep-th↗

arXiv2025

Hidden zeros for higher-derivative YM and GR amplitudes at tree-level

We extend the recently discovered phenomenon of hidden zeros to tree amplitudes for Yang-Mills (YM) and general relativity (GR) theories with higher-derivative interactions. This includes gluon amplitudes with a single insertion of the local $F^3$ operator, as well as graviton amplitudes at sub-leading and sub-sub-leading orders in the low-energy expansion of bosonic closed string amplitudes -- referred to as $R^2$ and $R^3$ amplitudes, respectively. The kinematic condition for hidden zeros leads to unavoidable propagator singularities in unordered graviton amplitudes. We investigate in detail the systematic cancellation of these divergences, which resolves ambiguities in the proof of hidden zeros. Our approach is based on universal expansions that express tree amplitudes as linear combinations of bi-adjoint scalar amplitudes.

hep-th↗

arXiv2025

IPR: Intelligent Prompt Routing with User-Controlled Quality-Cost Trade-offs

Routing incoming queries to the most cost-effective LLM while maintaining response quality poses a fundamental challenge in optimizing performance-cost trade-offs for large-scale commercial systems. We present IPR\, -- \,a quality-constrained \textbf{I}ntelligent \textbf{P}rompt \textbf{R}outing framework that dynamically selects optimal models based on predicted response quality and user-specified tolerance levels. IPR introduces three key innovations: (1) a modular architecture with lightweight quality estimators trained on 1.5M prompts annotated with calibrated quality scores, enabling fine-grained quality prediction across model families; (2) a user-controlled routing mechanism with tolerance parameter $τ\in [0,1]$ that provides explicit control over quality-cost trade-offs; and (3) an extensible design using frozen encoders with model-specific adapters, reducing new model integration from days to hours. To rigorously train and evaluate IPR, we curate an industrial-level dataset IPRBench\footnote{IPRBench will be released upon legal approval.}, a comprehensive benchmark containing 1.5 million examples with response quality annotations across 11 LLM candidates. Deployed on a major cloud platform, IPR achieves 43.9\% cost reduction while maintaining quality parity with the strongest model in the Claude family and processes requests with sub-150ms latency. The deployed system and additional product details are publicly available at https://aws.amazon.com/bedrock/intelligent-prompt-routing/

cs.LG↗

arXiv2025

$2$-split from Feynman diagrams and Expansions

In this paper, we investigate the $2$-split behavior of tree-level amplitudes of bi-adjoint scalar (BAS), Yang-Mills (YM), non-linear sigma model (NLSM), and general relativity (GR) theories under certain kinematic conditions. Our approach begins with a proof, based on the Feynman diagram method, of the $2$-split property for tree-level BAS$\oplus$X amplitudes with $\mathrm{X}={\mathrm{YM},\mathrm{NLSM},\mathrm{GR}}$. The proof relies crucially on a particular pattern in the Feynmam rules of various vertices. Building on this, we use the expansion of X amplitudes into BAS$\oplus$X amplitudes to establish the $2$-split behavior. As a byproduct, we derive universal expansions of the resulting pure X currents into BAS currents, which closely parallel the corresponding on-shell amplitude expansions.

hep-th↗

arXiv2025

Investigating Context-Faithfulness in Large Language Models: The Roles of Memory Strength and Evidence Style

Retrieval-augmented generation (RAG) improves Large Language Models (LLMs) by incorporating external information into the response generation process. However, how context-faithful LLMs are and what factors influence LLMs' context faithfulness remain largely unexplored. In this study, we investigate the impact of memory strength and evidence presentation on LLMs' receptiveness to external evidence. We quantify the memory strength of LLMs by measuring the divergence in LLMs' responses to different paraphrases of the same question, which is not considered by previous works. We also generate evidence in various styles to examine LLMs' behavior. Our results show that for questions with high memory strength, LLMs are more likely to rely on internal memory. Furthermore, presenting paraphrased evidence significantly increases LLMs' receptiveness compared to simple repetition or adding details. These findings provide key insights for improving retrieval-augmented generation and context-aware LLMs. Our code is available at https://github.com/liyp0095/ContextFaithful.

cs.CL↗

arXiv2025

Relation-Aware Network with Attention-Based Loss for Few-Shot Knowledge Graph Completion

Few-shot knowledge graph completion (FKGC) task aims to predict unseen facts of a relation with few-shot reference entity pairs. Current approaches randomly select one negative sample for each reference entity pair to minimize a margin-based ranking loss, which easily leads to a zero-loss problem if the negative sample is far away from the positive sample and then out of the margin. Moreover, the entity should have a different representation under a different context. To tackle these issues, we propose a novel Relation-Aware Network with Attention-Based Loss (RANA) framework. Specifically, to better utilize the plentiful negative samples and alleviate the zero-loss issue, we strategically select relevant negative samples and design an attention-based loss function to further differentiate the importance of each negative sample. The intuition is that negative samples more similar to positive samples will contribute more to the model. Further, we design a dynamic relation-aware entity encoder for learning a context-dependent entity representation. Experiments demonstrate that RANA outperforms the state-of-the-art models on two benchmark datasets.

cs.CL↗

arXiv2025

Bridge: A Unified Framework to Knowledge Graph Completion via Language Models and Knowledge Representation

Knowledge graph completion (KGC) is a task of inferring missing triples based on existing Knowledge Graphs (KGs). Both structural and semantic information are vital for successful KGC. However, existing methods only use either the structural knowledge from the KG embeddings or the semantic information from pre-trained language models (PLMs), leading to suboptimal model performance. Moreover, since PLMs are not trained on KGs, directly using PLMs to encode triples may be inappropriate. To overcome these limitations, we propose a novel framework called Bridge, which jointly encodes structural and semantic information of KGs. Specifically, we strategically encode entities and relations separately by PLMs to better utilize the semantic knowledge of PLMs and enable structured representation learning via a structural learning principle. Furthermore, to bridge the gap between KGs and PLMs, we employ a self-supervised representation learning method called BYOL to fine-tune PLMs with two different views of a triple. Unlike BYOL, which uses augmentation methods to create two semantically similar views of the same image, potentially altering the semantic information. We strategically separate the triple into two parts to create different views, thus avoiding semantic alteration. Experiments demonstrate that Bridge outperforms the SOTA models on three benchmark datasets.

cs.CL↗

arXiv2025

A Gradient Meta-Learning Joint Optimization for Beamforming and Antenna Position in Pinching-Antenna Systems

In this paper, we consider a novel optimization design for multi-waveguide pinching-antenna systems, aiming to maximize the weighted sum rate (WSR) by jointly optimizing beamforming coefficients and antenna position. To handle the formulated non-convex problem, a gradient-based meta-learning joint optimization (GML-JO) algorithm is proposed. Specifically, the original problem is initially decomposed into two sub-problems of beamforming optimization and antenna position optimization through equivalent substitution. Then, the convex approximation methods are used to deal with the nonconvex constraints of sub-problems, and two sub-neural networks are constructed to calculate the sub-problems separately. Different from alternating optimization (AO), where two sub-problems are solved alternately and the solutions are influenced by the initial values, two sub-neural networks of proposed GML-JO with fixed channel coefficients are considered as local sub-tasks and the computation results are used to calculate the loss function of joint optimization. Finally, the parameters of sub-networks are updated using the average loss function over different sub-tasks and the solution that is robust to the initial value is obtained. Simulation results demonstrate that the proposed GML-JO algorithm achieves 5.6 bits/s/Hz WSR within 100 iterations, yielding a 32.7\% performance enhancement over conventional AO with substantially reduced computational complexity. Moreover, the proposed GML-JO algorithm is robust to different choices of initialization and yields better performance compared with the existing optimization methods.

cs.IR↗

arXiv2025

MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttraining

We present MiMo-7B, a large language model born for reasoning tasks, with optimization across both pre-training and post-training stages. During pre-training, we enhance the data preprocessing pipeline and employ a three-stage data mixing strategy to strengthen the base model's reasoning potential. MiMo-7B-Base is pre-trained on 25 trillion tokens, with additional Multi-Token Prediction objective for enhanced performance and accelerated inference speed. During post-training, we curate a dataset of 130K verifiable mathematics and programming problems for reinforcement learning, integrating a test-difficulty-driven code-reward scheme to alleviate sparse-reward issues and employing strategic data resampling to stabilize training. Extensive evaluations show that MiMo-7B-Base possesses exceptional reasoning potential, outperforming even much larger 32B models. The final RL-tuned model, MiMo-7B-RL, achieves superior performance on mathematics, code and general reasoning tasks, surpassing the performance of OpenAI o1-mini. The model checkpoints are available at https://github.com/xiaomimimo/MiMo.

cs.CL↗

arXiv2025

MiMo-VL Technical Report

We open-source MiMo-VL-7B-SFT and MiMo-VL-7B-RL, two powerful vision-language models delivering state-of-the-art performance in both general visual understanding and multimodal reasoning. MiMo-VL-7B-RL outperforms Qwen2.5-VL-7B on 35 out of 40 evaluated tasks, and scores 59.4 on OlympiadBench, surpassing models with up to 78B parameters. For GUI grounding applications, it sets a new standard with 56.1 on OSWorld-G, even outperforming specialized models such as UI-TARS. Our training combines four-stage pre-training (2.4 trillion tokens) with Mixed On-policy Reinforcement Learning (MORL) integrating diverse reward signals. We identify the importance of incorporating high-quality reasoning data with long Chain-of-Thought into pre-training stages, and the benefits of mixed RL despite challenges in simultaneous multi-domain optimization. We also contribute a comprehensive evaluation suite covering 50+ tasks to promote reproducibility and advance the field. The model checkpoints and full evaluation suite are available at https://github.com/XiaomiMiMo/MiMo-VL.

cs.CL↗

arXiv2025

Soft theorems of tree-level ${\rm Tr}(ϕ^3)$, YM and NLSM amplitudes from $2$-splits

In this paper, we extend the method proposed in \cite{Arkani-Hamed:2024fyd} for deriving soft theorems of amplitudes, which relies exclusively on factorization properties including conventional factorizations on physical poles, as well as newly discovered $2$-splits on special loci in kinematic space. Using the extended approach, we fully reproduce the leading and sub-leading single-soft theorems for tree-level ${\rm Tr}(ϕ^3)$ and Yang-Mills (YM) amplitudes, along with the leading and sub-leading double-soft theorems for tree-level amplitudes of non-linear sigma model (NLSM). Furthermore, we establish universal representations of higher-order single-soft theorems for tree-level ${\rm Tr}(ϕ^3)$ and YM amplitudes in reduced lower-dimensional kinematic spaces. All obtained soft factors maintain consistency with momentum conservation; that is, while each explicit expression of the resulting soft behavior may changes under re-parameterization via momentum conservation, the physical content remains equivalent. Additionally, we find two interesting by-products: First, the single-soft theorems of YM amplitudes and the double-soft theorems of NLSM, at leading and sub-leading orders, are related by a simple kinematic replacement. This replacement also transmutes gauge invariance to Adler zero. Second, we obtain universal sub-leading soft theorems for the resulting pure YM and NLSM currents in the corresponding $2$-splits.

hep-th↗

arXiv2025

Towards a More Generalized Approach in Open Relation Extraction

Open Relation Extraction (OpenRE) seeks to identify and extract novel relational facts between named entities from unlabeled data without pre-defined relation schemas. Traditional OpenRE methods typically assume that the unlabeled data consists solely of novel relations or is pre-divided into known and novel instances. However, in real-world scenarios, novel relations are arbitrarily distributed. In this paper, we propose a generalized OpenRE setting that considers unlabeled data as a mixture of both known and novel instances. To address this, we propose MixORE, a two-phase framework that integrates relation classification and clustering to jointly learn known and novel relations. Experiments on three benchmark datasets demonstrate that MixORE consistently outperforms competitive baselines in known relation classification and novel relation clustering. Our findings contribute to the advancement of generalized OpenRE research and real-world applications.

cs.CL↗

arXiv2025

Black-Box Visual Prompt Engineering for Mitigating Object Hallucination in Large Vision Language Models

Large Vision Language Models (LVLMs) often suffer from object hallucination, which undermines their reliability. Surprisingly, we find that simple object-based visual prompting -- overlaying visual cues (e.g., bounding box, circle) on images -- can significantly mitigate such hallucination; however, different visual prompts (VPs) vary in effectiveness. To address this, we propose Black-Box Visual Prompt Engineering (BBVPE), a framework to identify optimal VPs that enhance LVLM responses without needing access to model internals. Our approach employs a pool of candidate VPs and trains a router model to dynamically select the most effective VP for a given input image. This black-box approach is model-agnostic, making it applicable to both open-source and proprietary LVLMs. Evaluations on benchmarks such as POPE and CHAIR demonstrate that BBVPE effectively reduces object hallucination.

cs.CV↗

arXiv2025

Hidden Zeros and $2$-split via BCFW Recursion Relation

In this paper, we provide another angle to understand recent discoveries, i.e., the hidden zeros and corresponding 2-split behavior using the BCFW recursion relation. For the hidden zeros, we show that although the BCFW recursion relation is not directly applicable for computing amplitudes of the non-linear sigma model, we can indeed prove the zeros using the modified BCFW recursion relation. Our work also indicates that for the 2-split to hold, the current should be carefully defined.

hep-th↗

arXiv2025

A Systematic Survey of Automatic Prompt Optimization Techniques

Since the advent of large language models (LLMs), prompt engineering has been a crucial step for eliciting desired responses for various Natural Language Processing (NLP) tasks. However, prompt engineering remains an impediment for end users due to rapid advances in models, tasks, and associated best practices. To mitigate this, Automatic Prompt Optimization (APO) techniques have recently emerged that use various automated techniques to help improve the performance of LLMs on various tasks. In this paper, we present a comprehensive survey summarizing the current progress and remaining challenges in this field. We provide a formal definition of APO, a 5-part unifying framework, and then proceed to rigorously categorize all relevant works based on their salient features therein. We hope to spur further research guided by our framework.

cs.CL↗