arXiv subjects

Min Cai

Publications and source records attributed to Min Cai.

18 recordsLinked to original sources

arXiv2026

GeoWAM: Visual Geometry World Action Models for Autonomous Driving

World action models (WAMs) have recently gained increasing attention as a framework for jointly modeling scene evolution and ego actions in autonomous driving. Most existing WAMs learn scene dynamics in pixel space by combining a video-generation backbone for future-observation prediction with an action head for ego-trajectory prediction. Pixels, however, provide only an indirect representation of these dynamics: they entangle geometry and motion with appearance, texture, and illumination, forcing the model to infer three-dimensional transformations from two-dimensional observations. We argue that geometry, represented by point clouds, offers a more natural state space for driving because it explicitly captures spatial structure and the rigid and non-rigid transformations that govern scene evolution while directly aligning with the space in which driving actions are executed. Building on this insight, we introduce \textbf{GeoWAM}, a visual geometry world action model for autonomous driving. Rather than predicting future images, GeoWAM is pretrained to forecast future scene geometry, yielding representations that jointly encode spatial structure and temporal evolution. A geometry-conditioned action head then leverages these learned geometric dynamics to predict future ego trajectories. Extensive open-loop and closed-loop evaluations show that visual geometry world modeling yields substantially stronger driving policies than image-based alternatives, establishing future-geometry prediction as an effective pretraining objective for autonomous driving.

cs.CV↗

arXiv2026

Tyler: Typed Latent Reasoning for Language Models -- When to Think, What to Compute, and How Much to Allocate

Chain-of-thought (CoT) prompting improves reasoning in large language models (LLMs) by externalizing intermediate computation as discrete text tokens, but this textual interface also introduces redundancy and inference overhead. Latent reasoning offers a promising alternative by carrying part of the computation in continuous representations. However, existing methods typically predefine when latent computation is invoked and how it is allocated during decoding, leaving a key problem unresolved: when to invoke latent computation, what type of computation to perform, and how much budget to allocate. We propose \textbf{Ty}ped \textbf{L}at\textbf{e}nt \textbf{R}easoning (Tyler), a typed and budget-aware framework for latent reasoning during autoregressive decoding. Tyler learns a policy that, at each decoding step, chooses between emitting a text token and switching to a latent computation module specialized for a particular reasoning function. Once invoked, an operator maps the current reasoning state into latent tokens that support global planning, local state updates, or reusable procedural abstraction. Across extensive experiments on three backbone LLMs, Tyler improves accuracy by up to 14.49 points over CoT and by up to 4.30 points over the strongest competing baseline. It further generalizes across diverse reasoning domains and achieves the best final-stage performance with the lowest forgetting.

cs.CL↗

arXiv2026

Advancing General-Purpose Reasoning Models with Modular Gradient Surgery

Reinforcement learning (RL) has played a central role in recent advances in large reasoning models (LRMs), yielding strong gains in verifiable and open-ended reasoning. However, training a single general-purpose LRM across diverse domains remains challenging due to pronounced domain heterogeneity. Through a systematic study of two widely used strategies, Sequential RL and Mixed RL, we find that both incur substantial cross-domain interference at the behavioral and gradient levels, resulting in limited overall gains. To address these challenges, we introduce **M**odular **G**radient **S**urgery (**MGS**), which resolves gradient conflicts at the module level within the transformer. When applied to Llama and Qwen models, MGS achieves average improvements of 4.3 (16.6\%) and 4.5 (11.1\%) points, respectively, over standard multi-task RL across three representative domains (math, general chat, and instruction following). Further analysis demonstrates that MGS remains effective under prolonged training. Overall, our study clarifies the sources of interference in multi-domain RL and presents an effective solution for training general-purpose LRMs.

cs.CL↗

arXiv2025

TDRM: Smooth Reward Models with Temporal Difference for LLM RL and Inference

Reward models are central to both reinforcement learning (RL) with language models and inference-time verification. However, existing reward models often lack temporal consistency, leading to ineffective policy updates and unstable RL training. We introduce TDRM, a method for learning smoother and more reliable reward models by minimizing temporal differences (TD) for training-time reinforcement learning and inference-time verification. Experiments show that TD-trained process reward models (PRMs) improve performance across Best-of-N (up to 6.6%) and tree-search (up to 23.7%) settings. When combined with Reinforcement Learning with Verifiable Rewards (RLVR), TD-trained PRMs lead to more data-efficient RL -- achieving comparable performance with just 2.5k data to what baseline methods require 50.1k data to attain -- and yield higher-quality language model policies in 8 model variants (5 series), e.g., Qwen2.5-(0.5B, 1,5B), GLM4-9B-0414, GLM-Z1-9B-0414, Qwen2.5-Math-(1.5B, 7B), and DeepSeek-R1-Distill-Qwen-(1.5B, 7B). We release all code at https://github.com/THUDM/TDRM.

cs.LG↗

arXiv2025

How Post-Training Reshapes LLMs: A Mechanistic View on Knowledge, Truthfulness, Refusal, and Confidence

Post-training is essential for the success of large language models (LLMs), transforming pre-trained base models into more useful and aligned post-trained models. While plenty of works have studied post-training algorithms and evaluated post-training models by their outputs, it remains understudied how post-training reshapes LLMs internally. In this paper, we compare base and post-trained LLMs mechanistically from four perspectives to better understand post-training effects. Our findings across model families and datasets reveal that: (1) Post-training does not change the factual knowledge storage locations, and it adapts knowledge representations from the base model while developing new knowledge representations; (2) Both truthfulness and refusal can be represented by vectors in the hidden representation space. The truthfulness direction is highly similar between the base and post-trained model, and it is effectively transferable for interventions; (3) The refusal direction is different between the base and post-trained models, and it shows limited forward transferability; (4) Differences in confidence between the base and post-trained models cannot be attributed to entropy neurons. Our study provides insights into the fundamental mechanisms preserved and altered during post-training, facilitates downstream tasks like model steering, and could potentially benefit future research in interpretability and LLM post-training. Our code is publicly available at https://github.com/HZD01/post-training-mechanistic-analysis.

cs.CL↗

arXiv2025

DataSciBench: An LLM Agent Benchmark for Data Science

This paper presents DataSciBench, a comprehensive benchmark for evaluating Large Language Model (LLM) capabilities in data science. Recent related benchmarks have primarily focused on single tasks, easily obtainable ground truth, and straightforward evaluation metrics, which limits the scope of tasks that can be evaluated. In contrast, DataSciBench is constructed based on a more comprehensive and curated collection of natural and challenging prompts for uncertain ground truth and evaluation metrics. We develop a semi-automated pipeline for generating ground truth (GT) and validating evaluation metrics. This pipeline utilizes and implements an LLM-based self-consistency and human verification strategy to produce accurate GT by leveraging collected prompts, predefined task types, and aggregate functions (metrics). Furthermore, we propose an innovative Task - Function - Code (TFC) framework to assess each code execution outcome based on precisely defined metrics and programmatic rules. Our experimental framework involves testing 6 API-based models, 8 open-source general models, and 9 open-source code generation models using the diverse set of prompts we have gathered. This approach aims to provide a more comprehensive and rigorous evaluation of LLMs in data science, revealing their strengths and weaknesses. Experimental results demonstrate that API-based models outperform open-sourced models on all metrics and Deepseek-Coder-33B-Instruct achieves the highest score among open-sourced models. We release all code and data at https://github.com/THUDM/DataSciBench.

cs.CL↗

arXiv2024

PIANIST: Learning Partially Observable World Models with LLMs for Multi-Agent Decision Making

Effective extraction of the world knowledge in LLMs for complex decision-making tasks remains a challenge. We propose a framework PIANIST for decomposing the world model into seven intuitive components conducive to zero-shot LLM generation. Given only the natural language description of the game and how input observations are formatted, our method can generate a working world model for fast and efficient MCTS simulation. We show that our method works well on two different games that challenge the planning and decision making skills of the agent for both language and non-language based action taking, without any training on domain-specific training data or explicitly defined world model.

cs.AI↗

arXiv2024

Strategist: Self-improvement of LLM Decision Making via Bi-Level Tree Search

Traditional reinforcement learning and planning typically requires vast amounts of data and training to develop effective policies. In contrast, large language models (LLMs) exhibit strong generalization and zero-shot capabilities, but struggle with tasks that require detailed planning and decision-making in complex action spaces. We introduce STRATEGIST, a novel approach that integrates the strengths of both methods. Our approach leverages LLMs to search and update high-level strategies (as text), which are then refined and executed by low-level Monte Carlo Tree Search (MCTS). STRATEGIST is a generalizable framework to optimize the strategy through population-based self-play simulations without the need for any training data. We demonstrate the effectiveness of STRATEGIST in learning optimal strategies for competitive, multi-turn games with partial information, including Game of Pure Strategy (GOPS) and multi-agent, hidden-identity discussion games like The Resistance: Avalon. Our results show that agents equipped with STRATEGIST outperform those trained with traditional RL methods, other LLM-based skill acquisition techniques, pre-existing LLM agents across both game environments and achieves comparable performance against human players.

cs.AI↗

arXiv2024

Self-Control of LLM Behaviors by Compressing Suffix Gradient into Prefix Controller

We propose SelfControl, an inference-time model control method utilizing gradients to control the behavior of large language models (LLMs) without explicit human annotations. Given a desired behavior expressed in a natural language suffix string concatenated to the input prompt, SelfControl computes gradients of the LLM's self-evaluation of the suffix with respect to its latent representations. The gradients are used to directly control the auto-regressive generation process towards desired behaviors, which eliminates human supervision, achieves precise and transparent control, and offers on-the-fly adaptability. To further enhance efficiency, we introduce SelfControl_{Prefix}, a compact module that encapsulates the learned representations from gradients into a SelfControl_{Prefix}, facilitating efficient inference-time control with no latency compared to the original model and allowing control for multiple behaviors simultaneously. Our experiments demonstrate SelfControl's efficacy across multiple domains, where it improves over SOTA for 8.3% in detoxification, 3.1% in truthfulness enhancement, 4%~10% in controlling on emotion tones, and 48.2% in privacy protection, i.e., completely remove privacy leakage issue. Additionally, we demonstrate that SelfControl can be used for data synthesis and to improve reasoning abilities.

cs.CL↗

arXiv2023

Self-Convinced Prompting: Few-Shot Question Answering with Repeated Introspection

While large language models (LLMs) such as ChatGPT and PaLM have demonstrated remarkable performance in various language understanding and generation tasks, their capabilities in complex reasoning and intricate knowledge utilization still fall short of human-level proficiency. Recent studies have established the effectiveness of prompts in steering LLMs towards generating desired outputs. Building on these insights, we introduce a novel framework that harnesses the potential of large-scale pre-trained language models, to iteratively enhance performance of the LLMs. Our framework incorporates three components: \textit{Normal CoT}, a \textit{Convincer}, and an \textit{Answerer}. It processes the output of a typical few-shot chain-of-thought prompt, assesses the correctness of the response, scrutinizes the answer, refines the reasoning, and ultimately produces a new solution. Experimental results on the 7 datasets of miscellaneous problems validate the efficacy of the Self-Convince framework, achieving substantial improvements compared to the baselines. This study contributes to the burgeoning body of research focused on integrating pre-trained language models with tailored prompts and iterative refinement processes to augment their performance in complex tasks.

cs.CL↗

arXiv2023

AvalonBench: Evaluating LLMs Playing the Game of Avalon

In this paper, we explore the potential of Large Language Models (LLMs) Agents in playing the strategic social deduction game, Resistance Avalon. Players in Avalon are challenged not only to make informed decisions based on dynamically evolving game phases, but also to engage in discussions where they must deceive, deduce, and negotiate with other players. These characteristics make Avalon a compelling test-bed to study the decision-making and language-processing capabilities of LLM Agents. To facilitate research in this line, we introduce AvalonBench - a comprehensive game environment tailored for evaluating multi-agent LLM Agents. This benchmark incorporates: (1) a game environment for Avalon, (2) rule-based bots as baseline opponents, and (3) ReAct-style LLM agents with tailored prompts for each role. Notably, our evaluations based on AvalonBench highlight a clear capability gap. For instance, models like ChatGPT playing good-role got a win rate of 22.2% against rule-based bots playing evil, while good-role bot achieves 38.2% win rate in the same setting. We envision AvalonBench could be a good test-bed for developing more advanced LLMs (with self-playing) and agent frameworks that can effectively model the layered complexities of such game environments.

cs.AI↗

arXiv2023

Two kinds of numerical algorithms for ultra-slow diffusion equations

In this article, two kinds of numerical algorithms are derived for the ultra-slow (or superslow) diffusion equation in one and two space dimensions, where the ultra-slow diffusion is characterized by the Caputo-Hadamard fractional derivative of order $\alpha \in (0,1)$. To describe the spatial interaction, the Riesz fractional derivative and the fractional Laplacian are used in one and two space dimensions, respectively. The Caputo-Hadamard derivative is discretized by two typical approximate formulae, i.e., L2-1$_{\sigma}$ and L1-2 methods. The spatial fractional derivatives are discretized by the 2-nd order finite difference methods. When L2-1$_{\sigma}$ discretization is used, the derived numerical scheme is unconditionally stable with error estimate $\mathcal{O}(\tau^{2}+h^{2})$ for all $\alpha \in (0, 1)$, in which $\tau$ and $h$ are temporal and spatial stepsizes, respectively. When L1-2 discretization is used, the derived numerical scheme is stable with error estimate $\mathcal{O}(\tau^{3-\alpha}+h^{2})$ for $\alpha \in (0, 0.3738)$. The illustrative examples displayed are in line with the theoretical analysis.

math.NA↗

arXiv2023

The Laplacian spectral ratio of connected graphs

Let $G$ be a simple connected undirected graph. The Laplacian spectral ratio of $G$, denoted by $R_L(G)$, is defined as the quotient between the largest and second smallest Laplacian eigenvalues of $G$, which is closely related to the structural parameters of a graph (or network), such as diameter, $t$-tough, perfect matching, average density of cuts, and synchronizability, etc. In this paper, we obtain some bounds of the Laplacian spectral ratio, which improves the known results. In addition, we give counter-examples on the upper bound of the Laplacian spectral ratio conjecture of trees, and propose a new conjecture.

math.CO↗

arXiv2022

Manipulating single excess electrons in monolayer transition metal dihalide

Polarons are entities of excess electrons dressed with local response of lattices, whose atomic-scale characterization is essential for understanding the many body physics arising from the electron-lattice entanglement, but yet difficult to achieve. Here, using scanning tunneling microscopy and spectroscopy (STM/STS), we show the visualization and manipulation of single polarons with different origin, i.e., electronic and conventional polarons, in monolayer CoCl2, that are grown on HOPG substrate via molecular beam epitaxy. Four types of polarons are identified, all inducing upward local band bending, but exhibiting distinct appearances, lattice occupations, polaronic states and local lattice distortions. First principles calculations unveil three types of polarons are stabilized by electron-electron interaction. The type-4 polaron, however, are driven by conventional lattice distortions. All the four types of polarons can be created, moved, erased, and moreover interconverted individually by the STM tip, allowing precise control of single polarons unprecedently. This finding identifies the rich category of polarons and their feasibility of manipulation in CoCl2, which can be generalized to other transition metal halides.

cond-mat.mes-hall↗

arXiv2022

Fractional SEIR Model and Data-Driven Predictions of COVID-19 Dynamics of Omicron Variant

We study the dynamic evolution of COVID-19 cased by the Omicron variant via a fractional susceptible-exposedinfected-removed (SEIR) model. Preliminary data suggest that the symptoms of Omicron infection are not prominent and the transmission is therefore more concealed, which causes a relatively slow increase in the detected cases of the new infected at the beginning of the pandemic. To characterize the specific dynamics, the Caputo-Hadamard fractional derivative is adopted to refined the classical SEIR model. Based on the reported data, we infer the fractional order, timedependent parameters, as well as unobserved dynamics of the fractional SEIR model via fractional physics-informed neural networks (fPINNs). Then, we make short-time predictions using the learned fractional SEIR model.

math.NA↗

arXiv2021

Small polarom formation by electron-electron interaction

In a solid, electrons can be scattered both by phonons and other electrons. First proposed by Landau, scattering by phonons can lead to a composite entity called a polaron, in which a lattice distortion traps an itinerant electron (or hole) such that the distortion and carrier move in unison as a single particle with larger effective mass. While this is the traditional view of polarons, the rise of 2D systems, especially strongly correlated ones, open the prospect of electron scattering taking on a larger role in spontaneous carrier localization for such material systems. Here, we show that in transition metal halides, such electron-electron interactions can lead to polaron formation even in the absence of lattice distortion. This suggests an alternative direction for polaron formation, transport, and control in solids. This new mechanism of polaron formation is confirmed by first-principles calculation of 2D transition metal halides, CrI2, CoCl2 and CoBr2. These theoretical predictions are supported by scanning tunneling microscopy/spectroscopy measurements of polarons in CrI2.

cond-mat.mtrl-sci↗

arXiv2021

Fractional Buffer Layers: Absorbing Boundary Conditions for Wave Propagation

We develop fractional buffer layers (FBLs) to absorb propagating waves without reflection in bounded domains. Our formulation is based on variable-order spatial fractional derivatives. We select a proper variable-order function so that dissipation is induced to absorb the coming waves in the buffer layers attached to the domain. In particular, we first design proper FBsL for the one-dimensional one-way and two-way wave propagation. Then, we extend our formulation to two-dimensional problems, where we introduce a consistent variable-order fractional wave equation. In each case, we obtain the fully discretized equations by employing a spectral collocation method in space and Crank-Nicolson or Adams-Bashforth method in time. We compare our results with the perfectly matched layer (PML) method and show the effectiveness of FBL in accurately suppressing any erroneously reflected waves, including corner reflections in two-dimensional rectangular domains. FBLs can be used in conjunction with any discretization method appropriate for fractional operators describing wave propagation in bounded or truncated domains.

math.NA↗

arXiv2019

Dimensional Crossover and Topological Phase Transition in Dirac Semimetal Na3Bi Films

Three-dimensional (3D) topological Dirac semimetal, when thinned down to 2D few layers, is expected to possess gapped Dirac nodes via quantum confinement effect and concomitantly display the intriguing quantum spin Hall (QSH) insulator phase. However, the 3D-to-2D crossover and the associated topological phase transition, which is valuable for understanding the topological quantum phases, remain unexplored. Here, we synthesize high-quality Na3Bi thin films with R3*R3 reconstruction on graphene, and systematically characterize their thickness-dependent electronic and topological properties by scanning tunneling microscopy/spectroscopy in combination with first-principles calculations. We demonstrate that Dirac gaps emerge in Na3Bi films, providing spectroscopic evidences of dimensional crossover from a 3D semimetal to a 2D topological insulator. Importantly, the Dirac gaps are revealed to be of sizable magnitudes on 3 and 4 monolayers (72 and 65 meV, respectively) with topologically nontrivial edge states. Moreover, the Fermi energy of a Na3Bi film can be tuned via certain growth process, thus offering a viable way for achieving charge neutrality in transport. The feasibility of controlling Dirac gap opening and charge neutrality enables realizing intrinsic high-temperature QSH effect in Na3Bi films and achieving potential applications in topological devices.

cond-mat.mtrl-sci↗