arXiv subjects

Jianlong Chen

Publications and source records attributed to Jianlong Chen.

At least 19 recordsLinked to original sources

arXiv2026

Diffusion Quasi-Monte Carlo

We study high-dimensional numerical integration with respect to complex target measures using diffusion-based transport maps and randomized quasi-Monte Carlo (RQMC). Score-based diffusion models induce a deterministic probability flow ODE that transports a simple prior to the target, suggesting a principled way to transform low-discrepancy points on the unit cube into informative samples. We construct a cube-to-target map by composing a Gaussian base transformation (the component-wise inverse Gaussian CDF) with an Euler-discretized probability flow ODE. To retain unbiasedness under transport approximation, we formulate integration as importance sampling (IS) on the cube. Our main result provides verifiable conditions under which the resulting IS integrand satisfies the boundary growth condition, implying an $O(N^{-1+\epsilon})$ RMSE for scrambled nets. We then establish these conditions for diffusion probability-flow transport under mild bounded-derivative assumptions on the learned vector field, explicitly controlling the boundary singularities introduced by the inverse Gaussian CDF. Experiments range from a 2D mixture to 784D images and a 40,960D conditional vorticity-assimilation task; in the latter, blocked scrambled Sobol' sampling reduces the randomization standard deviation of nonlinear accuracy metrics at essentially unchanged online denoising cost. Together, these results give a theoretical and empirical foundation for combining diffusion generative modeling with high-precision RQMC integration.

math.NA↗

arXiv2026

Kimi K3: Open Frontier Intelligence

We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Kimi K3 is built on Kimi Delta Attention and Attention Residuals, which improve information flow across sequence length and model depth. Together with Stable LatentMoE, which effectively activates 16 of 896 routed experts per token, and refined training and data recipes, these advances yield an approximately 2.5x improvement in overall scaling efficiency over Kimi K2. Post-training highlights reinforcement learning across general, agentic, and coding domains and multiple reasoning-effort levels, enabling compositional generalization and robust long-horizon execution. At 2.8T scale, Kimi K3 is supported by infrastructure advances in multiple areas: algorithm-system co-design for KDA, perfectly balanced expert-parallel training with efficient memory management, million-token agentic RL with persistent rollout and sandbox states, and deployment innovations. Extensive evaluations show that Kimi K3 achieves frontier-level performance across long-horizon coding, agentic, knowledge, reasoning, and vision tasks. While its overall performance still trails the most powerful proprietary models, namely Claude Fable 5 and GPT-5.6 Sol, Kimi K3 consistently outperforms other open and proprietary models evaluated in our suite. We release the full Kimi K3 model weights to facilitate future research and accelerate the broader deployment and adoption of frontier intelligence.

cs.CL↗

arXiv2026

GameCraft-Bench: Can Agents Build Playable Games End-to-End in a Real Game Engine?

Game generation is an emerging application of coding agents, requiring models to transform natural-language specifications into playable interactive systems. Unlike traditional coding tasks, game generation takes place within a game engine, where scripts, scenes, assets, rendering, and runtime interactions must jointly produce coherent gameplay. We formalize end-to-end game generation as the problem of producing a complete game artifact that realizes a specification through observable player-game interaction in a target environment. We argue that evaluating this setting requires three desiderata: Engine Grounding, Artifact Completeness, and Interactive Verification. We propose an interaction-grounded evaluation framework that assesses executable gameplay through replayed demonstrations and rubric-guided multimodal judging. We instantiate this framework as GameCraft-Bench, a benchmark comprising 140 Godot tasks across 15 game families. Evaluations of frontier coding agents show that end-to-end game generation remains highly challenging: the strongest agent achieves only 41.46%, and most agents score below 40%. Further analysis reveals that while agents often implement recognizable mechanics, they struggle to deliver complete games with sufficient content, functional visual feedback, and coherent presentation. See https://tongxuluo.github.io/gamecraft-bench-website for demos, code, and data.

cs.CL↗

arXiv2026

RealMath-Eval: Why SOTA Judges Struggle with Real Human Reasoning

While Large Language Models (LLMs) have achieved near-perfect performance in \emph{solving} high-school mathematics, their ability to \emph{evaluate} the diverse reasoning processes of real human students remains under-examined. To bridge this gap, we introduce \textbf{RealMath-Eval}, a rigorously annotated benchmark of 224 real-world exam responses from high schools. Our initial evaluation reveals that even state-of-the-art LLM judges struggle significantly on this task, exhibiting a high Mean Squared Error ($\sim$2.96) against expert human grading. To probe a plausible explanation, we contrast this performance with a control setting where the same judges evaluate synthetic LLM-generated solutions. We identify a stark ``Evaluation Gap'': judges are considerably more accurate and consistent on synthetic text (MSE $\sim$1.17) but struggle to generalize to authentic student reasoning. Through semantic embedding analysis, we find that synthetic errors suffer from a ``structural collapse'' into predictable, low-dimensional linear subspaces, whereas human errors form a more diverse error space. Furthermore, generative probability probes suggest that human reasoning involves significantly higher information-theoretic surprisal, indicating that student reasoning transitions are more out-of-distribution for current models. Finally, we find that surface-level style transfer fails to close this gap. Our findings suggest that current LLM evaluation pipelines relying heavily on synthetic data may not adequately capture the diversity of authentic student mathematical reasoning.

cs.AI↗

arXiv2026

TrustGeoGen: Formal-Verified Data Engine for Trustworthy Multi-modal Geometric Problem Solving

Geometric problem solving (GPS) requires precise multimodal understanding and rigorous, step-by-step logical reasoning. However, developing capable Multimodal Large Language Models (MLLMs) for GPS is heavily bottlenecked by the scarcity of high-quality, verifiable data. Existing data acquisition paradigms either suffer from modality incompleteness and unverified logical gaps ("leaps-of-faith"), or rely on formal engines that generate rigid, structurally homogeneous data, failing to produce high-difficulty problems or foster genuine natural-language reasoning. To overcome these limitations, we introduce TrustGeoGen, an autonomous and formalized geometric data generation engine. TrustGeoGen strictly guarantees reasoning trustworthiness through formal verification while generating multimodally integrated data, including premises, visual diagrams, and solutions. To systematically scale problem difficulty, we incorporates difficulty-aware filtering and iterative bootstrapping mechanism. Furthermore, we propose "connection thinking" to bridge the semantic gap between rigid formal logic and fluent human-like reasoning, ensuring coherent logical transitions. We also introduce the GeoExplore family of sampling algorithms to extract diverse problem-solving trajectories based on various thinking templates. Extensive experiments demonstrate that training models on our synthesized dataset, GeoTrust, substantially enhances deep geometric reasoning capabilities and yields significant performance gains across out-of-distribution (OOD) benchmarks, including GeoQA, Geometry3K, and OlympiadBench.Our code and data can be found at https://github.com/InternScience/TrustGeoGen

cs.AI↗

arXiv2026

MSSR: Memory-Aware Adaptive Replay for Continual LLM Fine-Tuning

Continual fine-tuning of large language models (LLMs) is becoming increasingly crucial as these models are deployed in dynamic environments where tasks and data distributions evolve over time. While strong adaptability enables rapid acquisition of new knowledge, it also exposes LLMs to catastrophic forgetting, where previously learned skills degrade during sequential training. Existing replay-based strategies, such as fixed interleaved replay, accuracy-supervised, and loss-driven scheduling, remain limited: some depend on heuristic rules and provide only partial mitigation of forgetting, while others improve performance but incur substantial computational overhead. Motivated by retention dynamics under sequential fine-tuning, we propose Memory-Inspired Sampler and Scheduler Replay (MSSR), an experience replay framework that estimates sample-level memory strength and schedules rehearsal at adaptive intervals to mitigate catastrophic forgetting while maintaining fast adaptation. Extensive experiments across three backbone models and 11 sequential tasks show that MSSR consistently outperforms state-of-the-art replay baselines, with particularly strong gains on reasoning-intensive and multiple-choice benchmarks.

cs.LG↗

arXiv2026

Revisiting Sharpness-Aware Minimization: A More Faithful and Effective Implementation

Sharpness-Aware Minimization (SAM) enhances generalization by minimizing the maximum training loss within a predefined neighborhood around the parameters. However, its practical implementation approximates this as gradient ascent(s) followed by applying the gradient at the ascent point to update the current parameters. This practice can be justified as approximately optimizing the objective by neglecting the (full) derivative of the ascent point with respect to the current parameters. Nevertheless, a direct and intuitive understanding of why using the gradient at the ascent point to update the current parameters works superiorly is still lacking. Our work bridges this gap by proposing a novel and intuitive interpretation. We show that the gradient at the single-step ascent point, \uline{when applied to the current parameters}, provides a better approximation of the direction from the current parameters toward the maximum within the local neighborhood than the local gradient. This improved approximation thereby enables a more direct escape from the maximum within the local neighborhood. Nevertheless, our analysis further reveals two issues. First, the approximation by the gradient at the single-step ascent point is often inaccurate. Second, the approximation quality may degrade as the number of ascent steps increases. To address these limitations, we propose in this paper eXplicit Sharpness-Aware Minimization (XSAM). It tackles the first by explicitly estimating the direction of the maximum during training, while addressing the second by crafting a search space that effectively leverages the gradient information at the multi-step ascent point. XSAM features a unified formulation that applies to both single-step and multi-step settings and only incurs negligible computational overhead. Extensive experiments demonstrate the consistent superiority of XSAM against existing counterparts.

cs.LG↗

arXiv2026

Epistemic Gain, Aleatoric Cost: Uncertainty Decomposition in Multi-Agent Debate for Math Reasoning

Multi-Agent Debate (MAD) has shown promise in improving reasoning and reducing hallucinations, yet it remains unclear how information exchange shapes individual reasoning behavior. Empirically, MAD exhibits paradoxical phenomena, including rising accuracy with increasing token entropy and marked differences between homogeneous and heterogeneous agent combinations. In this paper, we introduce a Bayesian uncertainty analysis framework for MAD, which decomposes answer-level predictive uncertainty into epistemic uncertainty and aleatoric uncertainty, corresponding to the potential gain and cost of debate. Across multiple agent configurations, we find that effective debate depends on achieving high epistemic gain under controlled aleatoric cost. Building on this insight, we design an uncertainty-guided multi-agent reinforcement learning algorithm that encourages lower aleatoric cost and more effective epistemic information utilization. Experiments show that our approach simultaneously enhances each agent's accuracy and promotes a more productive debate process, providing an operational Bayesian perspective for understanding and improving MAD.

cs.MA↗

arXiv2026

Kimi K2.5: Visual Agentic Intelligence

We introduce Kimi K2.5, an open-source multimodal agentic model designed to advance general agentic intelligence. K2.5 emphasizes the joint optimization of text and vision so that two modalities enhance each other. This includes a series of techniques such as joint text-vision pre-training, zero-vision SFT, and joint text-vision reinforcement learning. Building on this multimodal foundation, K2.5 introduces Agent Swarm, a self-directed parallel agent orchestration framework that dynamically decomposes complex tasks into heterogeneous sub-problems and executes them concurrently. Extensive evaluations show that Kimi K2.5 achieves state-of-the-art results across various domains including coding, vision, reasoning, and agentic tasks. Agent Swarm also reduces latency by up to $4.5\times$ over single-agent baselines. We release the post-trained Kimi K2.5 model checkpoint to facilitate future research and real-world applications of agentic intelligence.

cs.CL↗

arXiv2026

Milestones over Outcome: Unlocking Geometric Reasoning with Sub-Goal Verifiable Reward

Multimodal Large Language Models (MLLMs) struggle with complex geometric reasoning, largely because "black box" outcome-based supervision fails to distinguish between lucky guesses and rigorous deduction. To address this, we introduce a paradigm shift towards subgoal-level evaluation and learning. We first construct GeoGoal, a benchmark synthesized via a rigorous formal verification data engine, which converts abstract proofs into verifiable numeric subgoals. This structure reveals a critical divergence between reasoning quality and outcome accuracy. Leveraging this, we propose the Sub-Goal Verifiable Reward (SGVR) framework, which replaces sparse signals with dense rewards based on the Skeleton Rate. Experiments demonstrate that SGVR not only enhances geometric performance (+9.7%) but also exhibits strong generalization, transferring gains to general math (+8.0%) and other general reasoning tasks (+2.8%), demonstrating broad applicability across diverse domains.

cs.LG↗

arXiv2026

Flow Matching Transport for Quasi-Monte Carlo Integration

High-dimensional integration with respect to complex target measures remains a fundamental challenge in computational science. While Flow Matching (FM) offers a powerful paradigm for constructing continuous-time transport maps, its deployment in high-precision integration is severely limited by the discretization bias inherent to numerical ODE solvers and the lack of rigorous convergence guarantees when coupled with Quasi-Monte Carlo (QMC) methods. This paper addresses these critical gaps by proposing Flow Matching Importance Sampling Quasi-Monte Carlo (FM-ISQMC), a framework designed to transform biased generative flows into unbiased, high-order integration schemes. Methodologically, we construct a transport map by composing a logistic base transformation with an Euler-discretized neural ODE field and employ importance sampling to correct for residual transport errors. Our central contribution is twofold. First, we establish a general convergence analysis for QMC importance sampling with arbitrary transport maps, identifying sufficient growth conditions for the $\mathcal{O}(N^{-1+\varepsilon})$ root-mean-square error rate. Second, we rigorously prove that the specific transport architecture of Flow Matching satisfies these conditions. Consequently, we establish a $\mathcal{O}(N^{-1+\varepsilon})$ root-mean-square error for the unbiased FM-ISQMC estimator, extending classical QMC theory to the realm of generative models. Numerical experiments validate that FM-ISQMC consistently breaks through the error floor observed in direct transport methods, delivering superior precision. This work thus bridges the divide between deep generative modeling and numerical integration.

math.NA↗

arXiv2025

GeoBench: Rethinking Multimodal Geometric Problem-Solving via Hierarchical Evaluation

Geometric problem solving constitutes a critical branch of mathematical reasoning, requiring precise analysis of shapes and spatial relationships. Current evaluations of geometric reasoning in vision-language models (VLMs) face limitations, including the risk of test data contamination from textbook-based benchmarks, overemphasis on final answers over reasoning processes, and insufficient diagnostic granularity. To address these issues, we present GeoBench, a hierarchical benchmark featuring four reasoning levels in geometric problem-solving: Visual Perception, Goal-Oriented Planning, Rigorous Theorem Application, and Self-Reflective Backtracking. Through six formally verified tasks generated via TrustGeoGen, we systematically assess capabilities ranging from attribute extraction to logical error correction. Experiments reveal that while reasoning models like OpenAI-o3 outperform general MLLMs, performance declines significantly with increasing task complexity. Key findings demonstrate that sub-goal decomposition and irrelevant premise filtering critically influence final problem-solving accuracy, whereas Chain-of-Thought prompting unexpectedly degrades performance in some tasks. These findings establish GeoBench as a comprehensive benchmark while offering actionable guidelines for developing geometric problem-solving systems.

cs.CV↗

arXiv2025

Randomized Quasi-Monte Carlo and Importance Sampling for Super-Fast Growing Functions with Applications to Finance

Many problems can be formulated as high-dimensional integrals of discontinuous functions that exhibit significant boundary growth, challenging the error analysis and applications of randomized quasi-Monte Carlo (RQMC) methods. This paper studies RQMC methods for super-fast growing functions satisfying generalized exponential growth conditions, with a special focus on financial derivative pricing. The main contribution of this paper is threefold. First, by combining RQMC with importance sampling (IS), we derive a new error bound for a class of integrands, whose values and derivatives are bounded by the critical growth function $e^{A|\boldsymbol{x}|^2}$ with $A = 1/2$. This result extends the existing results in the literature, which are limited to the case $A < 1/2$. We demonstrate that by imposing a light-tailed condition on the proposal distribution of IS, RQMC can achieve an error rate of $O(n^{-1 + ε})$ with a sample size n and an arbitrarily small $ε>0$. Second, we verify that the Gaussian proposals used in Optimal Drift Importance Sampling (ODIS) satisfy the required light-tailed condition, providing a rigorous theoretical guarantees for RQMC-ODIS in critical growth scenarios. Third, for discontinuous integrands from finance, we prove that the integrands after preintegration satisfy the exponential growth condition. This ensures that the preintegrated functions can be seamlessly incorporated into our RQMC-IS framework. Numerical experiments on financial derivative pricing validate our theory, showing that the RQMC-IS with preintegration is effective in handling problems with discontinuous payoffs, successfully achieving the expected convergence rates.

math.NA↗

arXiv2025

New characterizations of weak CMP inverses

In 2025, Mosić defined the weak CMP inverse utilizing a minimal rank weak Drazin inverse instead of the Drazin inverse. The weak CMP inverse is a new wider class of generalized inverses, of which the CMP and MPCEP inverse are particular cases. In this paper, we provide several expressions, along with a number of new characterizations and properties for the weak CMP inverse. Moreover, we investigate the relationships between the weak CMP inverse and some well-known generalized inverses, such as the Moore-Penrose inverse and weak MPD inverse. Finally, we show that the weak CMP inverse, weak MPD inverse and weak DMP inverse are all strong Bott-Duffin $(e,f)$-inverses.

math.RA↗

arXiv2025

Additive properties and absorption laws for generalized inverses

Let $a,~f$ be elements in a ring with pseudo core inverses $a^{\scriptsize\textcircled{\tiny D}}$, $f^{\scriptsize\textcircled{\tiny D}}$, and let $b=f-a$. We prove that the absorption law $a^{\scriptsize\textcircled{\tiny D}}(a+f)f^{\scriptsize\textcircled{\tiny D}}=a^{\scriptsize\textcircled{\tiny D}}+f^{\scriptsize\textcircled{\tiny D}}$ holds if and only if $1+a^{\scriptsize\textcircled{\tiny D}}b$ is invertible and the additive property $f^{\scriptsize\textcircled{\tiny D}}=(1+a^{\scriptsize\textcircled{\tiny D}}b)^{-1}a^{\scriptsize\textcircled{\tiny D}}$ is satisfied. We further characterize these properties and establish analogous results for other generalized inverses. Finally, we apply these results to the case of complex matrices.

math.RA↗

arXiv2025

Enhanced convergence rates of Adaptive Importance Sampling with recycling schemes via quasi-Monte Carlo methods

This article investigates the integration of quasi-Monte Carlo (QMC) methods using the Adaptive Multiple Importance Sampling (AMIS). Traditional Importance Sampling (IS) often suffers from poor performance since it heavily relies on the choice of the proposal distributions. The AMIS and the Modified version of AMIS (MAMIS) address this by iteratively refining proposal distributions and reusing all past samples through a recycling strategy. We introduce the RQMC methods into the MAMIS, achieving higher convergence rates compared to the Monte Carlo (MC) methods. Our main contributions include a detailed convergence analysis of the MAMIS estimator under randomized QMC (RQMC) sampling. Specifically, we establish the $L^q$ $(q \geq 2)$ error bound for the RQMC-based estimator using a smoothed projection method, which enables us to apply the Hölder's inequality in the error analysis of the RQMC-based MAMIS estimator. As a result, we prove that the root mean square error of the RQMC-based MAMIS estimator converges at a rate of $\mathcal{O}(\bar{N}_T^{-1+ε})$, where $\bar{N}_T$ is the average number of samples used in each step over $T$ iterations, and $ε> 0$ is arbitrarily small. Numerical experiments validate the effectiveness of our method, including mixtures of Gaussians, a banana-shaped model, and Bayesian Logistic regression.

math.NA↗

arXiv2025

Hierarchical Attention Generates Better Proofs

Large language models (LLMs) have shown promise in formal theorem proving, but their token-level processing often fails to capture the inherent hierarchical nature of mathematical proofs. We introduce \textbf{Hierarchical Attention}, a regularization method that aligns LLMs' attention mechanisms with mathematical reasoning structures. Our approach establishes a five-level hierarchy from foundational elements to high-level concepts, ensuring structured information flow in proof generation. Experiments demonstrate that our method improves proof success rates by 2.05\% on miniF2F and 1.69\% on ProofNet while reducing proof complexity by 23.81\% and 16.50\% respectively. The code is available at https://github.com/Car-pe/HAGBP.

cs.LG↗

arXiv2024

Leveraging Large Language Models to Enhance Personalized Recommendations in E-commerce

This study deeply explores the application of large language model (LLM) in personalized recommendation system of e-commerce. Aiming at the limitations of traditional recommendation algorithms in processing large-scale and multi-dimensional data, a recommendation system framework based on LLM is proposed. Through comparative experiments, the recommendation model based on LLM shows significant improvement in multiple key indicators such as precision, recall, F1 score, average click-through rate (CTR) and recommendation diversity. Specifically, the precision of the LLM model is improved from 0.75 to 0.82, the recall rate is increased from 0.68 to 0.77, the F1 score is increased from 0.71 to 0.79, the CTR is increased from 0.56 to 0.63, and the recommendation diversity is increased by 41.2%, from 0.34 to 0.48. LLM effectively captures the implicit needs of users through deep semantic understanding of user comments and product description data, and combines contextual data for dynamic recommendation to generate more accurate and diverse results. The study shows that LLM has significant advantages in the field of personalized recommendation, can improve user experience and promote platform sales growth, and provides strong theoretical and practical support for personalized recommendation technology in e-commerce.

cs.IR↗