arXiv subjects

Bowen Shen

Publications and source records attributed to Bowen Shen.

At least 19 recordsLinked to original sources

arXiv2026

Can Large Language Models Resolve Real Java Merge Conflicts? An Evaluation with a Calibrated LLM-as-Judge

Merge conflicts are a recurring cost of collaborative software development, and the traditional structured and semi-structured merge tools that address them frequently abstain: when their heuristics do not apply, they leave the conflict unresolved. Large language models (LLMs) can instead produce a candidate resolution for almost any conflict, but measuring whether those resolutions are actually good at scale is hard, because obtaining human desirability judgments for every model output does not scale. We study both problems together on real Java merge conflicts from ConflictBench. We first build an LLM solver as a generate-validate-retry agent that uses only inference-time signals (conflict markers, a Java parser, and duplicate-declaration checks) and never sees the developer's answer. We then evaluate its resolutions with a two-metric suite: (1) a developer-match LLM-as-judge implemented as a G-Eval metric and, crucially, calibrated against ConflictBench's human labels before use, and (2) a deterministic structural-validity check that uses no LLM. On a meta-evaluation of 292 human-labeled cases, the judge reaches 100% precision (zero false accepts) at 64.6% recall, so every acceptance is trustworthy and every downstream rate is a conservative lower bound. Under this validated judge, LLM solvers match the developer's own resolution on about 55% of true conflicts (conservative floor), and under a coverage-fair comparison the LLMs (55-59%) beat the strongest traditional tool (AutoMerge, 36.7%) by roughly 18-22 points; the edge comes almost entirely from coverage, not raw accuracy, since the tools abstain on 20-90% of conflicts while the LLM under forced resolution abstains on none. Finally, the LLM judge accepted 4 of the 5 resolutions that fail the deterministic structural check, evidence that structural correctness must not be delegated to an LLM.

cs.SE↗

arXiv2026

MemMorph: Tool Hijacking in LLM Agents via Memory Poisoning

LLM-driven agents are capable of selecting external tools to complete users' tasks. However, attackers could compromise such process, steering agents toward inappropriate/wrong tools and enabling malicious actions. Most existing attacks primarily manipulate the tool metadata, which is easily detectable by auditing and may lose effectiveness as modern agents increasingly adopt memory modules to refine tool selection policies through accumulated experience. This paper proposes MemMorph, the first attack that bias tool selection by poisoning the agent's long-term memory. Rather than explicitly dictating the tool invocation decision, MemMorph injects a small number of crafted records that are disguised as technical facts, incident reports, and operational policies. These poisoned records reshape the agent's contextual perception and decision-making process, leading it to autonomously infer and select the tool preferred by the attacker. Experiments across 3 benchmarks, 10 agent backbones, and 3 memory-module implementations show that MemMorph achieves up to 85.9% attack success rate with only three injected records, outperforming the strongest baseline by up to 25% while retaining potency under 3 representative defenses. Our findings expose long-term memory as a critical and under-explored attack surface in tool-augmented agents, urging the development of memory-level integrity safeguards.

cs.CR↗

arXiv2026

Stacking-order-dependent electronic properties of MoTe2/WSe2 moir\'e bilayers

Transition metal dichalcogenide (TMD) moir\'e bilayers have realized a wide range of strongly correlated and topological phenomena. The physics in these materials is often sensitive to the interlayer stacking order. Polarization-resolved optical second harmonic generation (SHG) is the most used technique for stacking order characterization but unverified for most heterobilayers. Here we calibrate the optical SHG for angle-aligned MoTe2/WSe2 bilayers by the scanning transmission electron microscopy (STEM). We directly compare the transport and magnetic properties and the electronic phase diagram for two distinct stacking orders. With the calibrated stacking order assignment, we clarify the interpretation of earlier results, including the nature of the Chern insulator, mechanism of an electric-field-tuned metal-insulator transition at half band filling, and the Kondo lattice physics. Our work provides a consistent picture of the relation between the stacking order and the electronic properties of MoTe2/WSe2 moir\'e bilayers.

cond-mat.mes-hall↗

arXiv2026

Emergence of a symmetry-broken Chern insulator near a moir\'e Kondo breakdown

Moir\'e semiconductors built on angle-aligned transition metal dichalcogenide (TMD) heterobilayers provide a physical realization of the Kondo lattice model, in which one TMD layer is prepared in a Mott insulating state supporting a lattice of local magnetic moments and the other layer in a metallic state supporting itinerant carriers. The artificial Kondo lattice enables the exploration of exotic states of matter near a continuously tunable Kondo breakdown. Here we report the emergence of a symmetry-broken Chern insulator at a moir\'e hole filling factor 4/3 in angle-aligned MoTe2/WSe2 moir\'e bilayers, which realize a chiral Kondo lattice. The symmetry-broken Chern insulator, which exhibits integer quantized Hall conductance at a fractional moir\'e filling, breaks the translational symmetry of the lattice spontaneously; it also appears only near a magnetic field-induced Kondo breakdown in the mixed-valence regime of the material. We further demonstrate that the magnetic field required to induce the Kondo breakdown and to stabilize the symmetry-broken Chern insulator is twist angle dependent. The results present new opportunities for exploring the subtle interplay between topology and Kondo interactions in moir\'e semiconductors.

cond-mat.mes-hall↗

arXiv2026

Electric-field-tuned consecutive topological phase transitions between distinct correlated insulators in moire MoTe2/WSe2 heterobilayer

Consecutive topological phase transitions (TPTs) between strongly correlated electronic phases that differ simultaneously in symmetry breaking and topological order are of fundamental interest in condensed matter physics, yet are rarely realized experimentally. We report two consecutive electric-field-driven TPTs at half filling (nu = 1) in angle-aligned MoTe2/WSe2 moire heterobilayers. With increasing out-of-plane displacement field, a geometrically frustrated Mott insulator evolves into a ferromagnetic quantum anomalous Hall (QAH) Mott insulator, i.e., a spin-polarized topological Mott insulator without an observable charge-gap closure, and subsequently into an antiferromagnetic, valley-coherent Mott insulator (VC-AFM) accompanied by a continuous charge-gap collapse and the emergence of a critical metallic state. Layer-resolved magnetic circular dichroism (MCD), magneto-transport, and compressibility measurements jointly determine the phase diagram. The high-field evolution of the antiferromagnetic state reveals a metamagnetic-like transition at a critical field B*, above which a Chern insulating transport response reappears. Our results establish the MoTe2/WSe2 moire platform as a tunable realization of an extended Kane-Mele-Hubbard model hosting sequential correlation-topology-intertwined transitions.

cond-mat.str-el↗

arXiv2026

SecMoE: Communication-Efficient Secure MoE Inference via Select-Then-Compute

Privacy-preserving Transformer inference has gained attention due to the potential leakage of private information. Despite recent progress, existing frameworks still fall short of practical model scales, with gaps up to a hundredfold. A possible way to close this gap is the Mixture of Experts (MoE) architecture, which has emerged as a promising technique to scale up model capacity with minimal overhead. However, given that the current secure two-party (2-PC) protocols allow the server to homomorphically compute the FFN layer with its plaintext model weight, under the MoE setting, this could reveal which expert is activated to the server, exposing token-level privacy about the client's input. While naively evaluating all the experts before selection could protect privacy, it nullifies MoE sparsity and incurs the heavy computational overhead that sparse MoE seeks to avoid. To address the privacy and efficiency limitations above, we propose a 2-PC privacy-preserving inference framework, \SecMoE. Unifying per-entry circuits in both the MoE layer and piecewise polynomial functions, \SecMoE obliviously selects the extracted parameters from circuits and only computes one encrypted entry, which we refer to as Select-Then-Compute. This makes the model for private inference scale to 63$\times$ larger while only having a 15.2$\times$ increase in end-to-end runtime. Extensive experiments show that, under 5 expert settings, \SecMoE lowers the end-to-end private inference communication by 1.8$\sim$7.1$\times$ and achieves 1.3$\sim$3.8$\times$ speedup compared to the state-of-the-art (SOTA) protocols.

cs.CR↗

arXiv2026

MiMo-V2-Flash Technical Report

We present MiMo-V2-Flash, a Mixture-of-Experts (MoE) model with 309B total parameters and 15B active parameters, designed for fast, strong reasoning and agentic capabilities. MiMo-V2-Flash adopts a hybrid attention architecture that interleaves Sliding Window Attention (SWA) with global attention, with a 128-token sliding window under a 5:1 hybrid ratio. The model is pre-trained on 27 trillion tokens with Multi-Token Prediction (MTP), employing a native 32k context length and subsequently extended to 256k. To efficiently scale post-training compute, MiMo-V2-Flash introduces a novel Multi-Teacher On-Policy Distillation (MOPD) paradigm. In this framework, domain-specialized teachers (e.g., trained via large-scale reinforcement learning) provide dense and token-level reward, enabling the student model to perfectly master teacher expertise. MiMo-V2-Flash rivals top-tier open-weight models such as DeepSeek-V3.2 and Kimi-K2, despite using only 1/2 and 1/3 of their total parameters, respectively. During inference, by repurposing MTP as a draft model for speculative decoding, MiMo-V2-Flash achieves up to 3.6 acceptance length and 2.6x decoding speedup with three MTP layers. We open-source both the model weights and the three-layer MTP weights to foster open research and community collaboration.

cs.CL↗

arXiv2025

MiMo-Audio: Audio Language Models are Few-Shot Learners

Existing audio language models typically rely on task-specific fine-tuning to accomplish particular audio tasks. In contrast, humans are able to generalize to new audio tasks with only a few examples or simple instructions. GPT-3 has shown that scaling next-token prediction pretraining enables strong generalization capabilities in text, and we believe this paradigm is equally applicable to the audio domain. By scaling MiMo-Audio's pretraining data to over one hundred million of hours, we observe the emergence of few-shot learning capabilities across a diverse set of audio tasks. We develop a systematic evaluation of these capabilities and find that MiMo-Audio-7B-Base achieves SOTA performance on both speech intelligence and audio understanding benchmarks among open-source models. Beyond standard metrics, MiMo-Audio-7B-Base generalizes to tasks absent from its training data, such as voice conversion, style transfer, and speech editing. MiMo-Audio-7B-Base also demonstrates powerful speech continuation capabilities, capable of generating highly realistic talk shows, recitations, livestreaming and debates. At the post-training stage, we curate a diverse instruction-tuning corpus and introduce thinking mechanisms into both audio understanding and generation. MiMo-Audio-7B-Instruct achieves open-source SOTA on audio understanding benchmarks (MMSU, MMAU, MMAR, MMAU-Pro), spoken dialogue benchmarks (Big Bench Audio, MultiChallenge Audio) and instruct-TTS evaluations, approaching or surpassing closed-source models. Model checkpoints and full evaluation suite are available at https://github.com/XiaomiMiMo/MiMo-Audio.

cs.CL↗

arXiv2025

Emergence of Chern metal in a moiré Kondo lattice

A Chern metal is a two-dimensional metallic state of matter carrying chiral edge states. It can emerge as a doped Chern insulator, but theoretical studies have also predicted its emergence near a Kondo breakdown separating a metallic chiral spin liquid and a heavy Fermi liquid in a frustrated lattice. To date, the latter exotic scenario has not been realized. Here, we report the observation of a Chern metal at the onset of the magnetic Kondo breakdown in a frustrated moiré Kondo lattice--angle-aligned MoTe2/WSe2 bilayers. The state is compressible and is manifested by a nearly quantized Hall resistance but a finite longitudinal resistance that arises from a bad metallic bulk. The state also separates an itinerant and a heavy Fermi liquid and appears far away from the band inversion critical point of the material, thus ruling out its origin from simply doping a Chern insulator. We demonstrate the presence of a chiral edge state by nonlocal transport measurements and current-induced quantum anomalous Hall breakdown. Magnetic circular dichroism measurements further reveal a magnetization plateau for the Chern metal before a metamagnetic transition at the Kondo breakdown. Our results open an opportunity for moiré engineering of exotic quantum phases of matter through the close interplay between band topology and Kondo interactions.

cond-mat.str-el↗

arXiv2025

Privacy-Aware Spectrum Pricing and Power Control Optimization for LEO Satellite Internet-of-Things

Low earth orbit (LEO) satellite systems play an important role in next generation communication networks due to their ability to provide extensive global coverage with guaranteed communications in remote areas and isolated areas where base stations cannot be cost-efficiently deployed. With the pervasive adoption of LEO satellite systems, especially in the LEO Internet-of-Things (IoT) scenarios, their spectrum resource management requirements have become more complex as a result of massive service requests and high bandwidth demand from terrestrial terminals. For instance, when leasing the spectrum to terrestrial users and controlling the uplink transmit power, satellites collect user data for machine learning purposes, which usually are sensitive information such as location, budget and quality of service (QoS) requirement. To facilitate model training in LEO IoT while preserving the privacy of data, blockchain-driven federated learning (FL) is widely used by leveraging on a fully decentralized architecture. In this paper, we propose a hybrid spectrum pricing and power control framework for LEO IoT by combining blockchain technology and FL. We first design a local deep reinforcement learning algorithm for LEO satellite systems to learn a revenue-maximizing pricing and power control scheme. Then the agents collaborate to form a FL system. We also propose a reputation-based blockchain which is used in the global model aggregation phase of FL. Based on the reputation mechanism, a node is selected for each global training round to perform model aggregation and block generation, which can further enhance the decentralization of the network and guarantee the trust. Simulation tests are conducted to evaluate the performances of the proposed scheme. Our results show the efficiency of finding the maximum revenue scheme for LEO satellite systems while preserving the privacy of each agent.

cs.NI↗

arXiv2025

DIVE into MoE: Diversity-Enhanced Reconstruction of Large Language Models from Dense into Mixture-of-Experts

Large language models (LLMs) with the Mixture-of-Experts (MoE) architecture achieve high cost-efficiency by selectively activating a subset of the parameters. Despite the inference efficiency of MoE LLMs, the training of extensive experts from scratch incurs substantial overhead, whereas reconstructing a dense LLM into an MoE LLM significantly reduces the training budget. However, existing reconstruction methods often overlook the diversity among experts, leading to potential redundancy. In this paper, we come up with the observation that a specific LLM exhibits notable diversity after being pruned on different calibration datasets, based on which we present a Diversity-Enhanced reconstruction method named DIVE. The recipe of DIVE includes domain affinity mining, pruning-based expert reconstruction, and efficient retraining. Specifically, the reconstruction includes pruning and reassembly of the feed-forward network (FFN) module. After reconstruction, we efficiently retrain the model on routers, experts and normalization modules. We implement DIVE on Llama-style LLMs with open-source training corpora. Experiments show that DIVE achieves training efficiency with minimal accuracy trade-offs, outperforming existing pruning and MoE reconstruction methods with the same number of activated parameters.

cs.CL↗

arXiv2025

MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttraining

We present MiMo-7B, a large language model born for reasoning tasks, with optimization across both pre-training and post-training stages. During pre-training, we enhance the data preprocessing pipeline and employ a three-stage data mixing strategy to strengthen the base model's reasoning potential. MiMo-7B-Base is pre-trained on 25 trillion tokens, with additional Multi-Token Prediction objective for enhanced performance and accelerated inference speed. During post-training, we curate a dataset of 130K verifiable mathematics and programming problems for reinforcement learning, integrating a test-difficulty-driven code-reward scheme to alleviate sparse-reward issues and employing strategic data resampling to stabilize training. Extensive evaluations show that MiMo-7B-Base possesses exceptional reasoning potential, outperforming even much larger 32B models. The final RL-tuned model, MiMo-7B-RL, achieves superior performance on mathematics, code and general reasoning tasks, surpassing the performance of OpenAI o1-mini. The model checkpoints are available at https://github.com/xiaomimimo/MiMo.

cs.CL↗

arXiv2025

MiMo-VL Technical Report

We open-source MiMo-VL-7B-SFT and MiMo-VL-7B-RL, two powerful vision-language models delivering state-of-the-art performance in both general visual understanding and multimodal reasoning. MiMo-VL-7B-RL outperforms Qwen2.5-VL-7B on 35 out of 40 evaluated tasks, and scores 59.4 on OlympiadBench, surpassing models with up to 78B parameters. For GUI grounding applications, it sets a new standard with 56.1 on OSWorld-G, even outperforming specialized models such as UI-TARS. Our training combines four-stage pre-training (2.4 trillion tokens) with Mixed On-policy Reinforcement Learning (MORL) integrating diverse reward signals. We identify the importance of incorporating high-quality reasoning data with long Chain-of-Thought into pre-training stages, and the benefits of mixed RL despite challenges in simultaneous multi-domain optimization. We also contribute a comprehensive evaluation suite covering 50+ tasks to promote reproducibility and advance the field. The model checkpoints and full evaluation suite are available at https://github.com/XiaomiMiMo/MiMo-VL.

cs.CL↗

arXiv2025

TailorKV: A Hybrid Framework for Long-Context Inference via Tailored KV Cache Optimization

The Key-Value (KV) cache in generative large language models (LLMs) introduces substantial memory overhead. Existing works mitigate this burden by offloading or compressing the KV cache. However, loading the entire cache incurs significant latency due to PCIe bandwidth bottlenecks in CPU-GPU communication, while aggressive compression causes notable performance degradation. We identify that certain layers in the LLM need to maintain global information and are unsuitable for selective loading. In contrast, other layers primarily focus on a few tokens with dominant activations that potentially incur substantial quantization error. This observation leads to a key insight that loading dominant tokens and quantizing all tokens can complement each other. Building on this insight, we propose a hybrid compression method, TailorKV, which seamlessly integrates quantization and offloading. TailorKV develops an inference framework along with a hardware-friendly implementation that leverages these complementary characteristics. Extensive long-context evaluations exhibit that TailorKV achieves nearly lossless performance under aggressive compression settings, outperforming the state-of-the-art. Particularly, the Llama-3.1-8B with 128k context can be served within a single RTX 3090 GPU, reaching 82 ms per token during decoding.

cs.CL↗

arXiv2025

Time-reversal symmetry breaking fractional quantum spin Hall insulator in moiré MoTe2

Twisted bilayer transition metal dichalcogenide semiconductors, which support flat Chern bands with enhanced interaction effects, realize a platform for fractional Chern insulators and fractional quantum spin Hall (FQSH) insulators. A recent experiment has reported the emergence of a FQSH insulator protected by spin-Sz conservation at a moiré lattice filling factor ν=3 in 2.1-degree twisted bilayer MoTe2. Theoretical studies have proposed both time-reversal symmetric and asymmetric ground states as possible candidates for the observed FQSH insulator, but the nature of the state remains unexplored. Here we report the observation of spontaneous time-reversal symmetry breaking at generic fillings in 2.1-degree twisted bilayer MoTe2 from ν<1 all the way to ν>6 except at ν=2, 4, and 6. Although zero Hall response is observed at ν=3 for magnetic fields higher than 20 mT, a finite anomalous Hall response accompanied by a magnetic hysteresis is observed at lower magnetic fields, demonstrating spontaneous time-reversal symmetry breaking. Our work shows the tendency towards ferromagnetism by doping the first three pairs of conjugate Chern bands in the material; it also sheds light on the nature of the FQSH insulator at ν=3.

cond-mat.mes-hall↗

arXiv2024

Pruning Large Language Models to Intra-module Low-rank Architecture with Transitional Activations

Structured pruning fundamentally reduces computational and memory overheads of large language models (LLMs) and offers a feasible solution for end-side LLM deployment. Structurally pruned models remain dense and high-precision, highly compatible with further tuning and compression. However, as the coarse-grained structured pruning poses large damage to the highly interconnected model, achieving a high compression ratio for scaled-up LLMs remains a challenge. In this paper, we introduce a task-agnostic structured pruning approach coupled with a compact Transformer architecture design. The proposed approach, named TransAct, reduces transitional activations inside multi-head attention (MHA) and multi-layer perceptron (MLP) modules, while preserving the inter-module activations that are sensitive to perturbations. Hence, the LLM is pruned into an intra-module low-rank architecture, significantly reducing weights, KV Cache and attention computation. TransAct is implemented on the LLaMA model and evaluated on downstream benchmarks. Results verify the optimality of our approach at high compression with respect to both efficiency and performance. Further, ablation studies reveal the strength of activation-guided iterative pruning and provide experimental analysis on the redundancy of MHA and MLP modules.

cs.CL↗

arXiv2024

Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning

Parameter-efficient fine-tuning (PEFT) has emerged as the predominant technique for fine-tuning in the era of large language models. However, existing PEFT methods still have inadequate training efficiency. Firstly, the utilization of large-scale foundation models during the training process is excessively redundant for certain fine-tuning tasks. Secondly, as the model size increases, the growth in trainable parameters of empirically added PEFT modules becomes non-negligible and redundant, leading to inefficiency. To achieve task-specific efficient fine-tuning, we propose the Light-PEFT framework, which includes two methods: Masked Early Pruning of the Foundation Model and Multi-Granularity Early Pruning of PEFT. The Light-PEFT framework allows for the simultaneous estimation of redundant parameters in both the foundation model and PEFT modules during the early stage of training. These parameters can then be pruned for more efficient fine-tuning. We validate our approach on GLUE, SuperGLUE, QA tasks, and various models. With Light-PEFT, parameters of the foundation model can be pruned by up to over 40%, while still controlling trainable parameters to be only 25% of the original PEFT method. Compared to utilizing the PEFT method directly, Light-PEFT achieves training and inference speedup, reduces memory usage, and maintains comparable performance and the plug-and-play feature of PEFT.

cs.CL↗

arXiv2024

Observation of the fractional quantum spin Hall effect in moiré MoTe2

Quantum spin Hall (QSH) insulators are two-dimensional electronic materials that have a bulk band gap like an ordinary insulator but have topologically protected pairs of edge modes of opposite chiralities. To date, experimental studies have found only integer QSH insulators with counter-propagating up-spins and down-spins at each edge leading to a quantized conductance G0=e^2/h. Here we report transport evidence of a fractional QSH insulator in 2.1-degree-twisted bilayer MoTe2, which supports spin-Sz conservation and flat spin-contrasting Chern bands. At filling factor v = 3 of the moiré valence bands, each edge contributes a conductance 3/2 G0 with zero anomalous Hall conductivity. The state is likely a time-reversal pair of the even-denominator 3/2-fractional Chern insulators. Further, at v = 2, 4 and 6, we observe a single, double and triple QSH insulator with each edge contributing a conductance G0, 2G0 and 3G0, respectively. Our results open up the possibility of realizing time reversal symmetric non-abelian anyons and other unexpected topological phases in highly tunable moiré materials.

cond-mat.mes-hall↗