arXiv subjects

Ziwei Chai

Publications and source records attributed to Ziwei Chai.

13 recordsLinked to original sources

arXiv2026

Kimi K3: Open Frontier Intelligence

We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Kimi K3 is built on Kimi Delta Attention and Attention Residuals, which improve information flow across sequence length and model depth. Together with Stable LatentMoE, which effectively activates 16 of 896 routed experts per token, and refined training and data recipes, these advances yield an approximately 2.5x improvement in overall scaling efficiency over Kimi K2. Post-training highlights reinforcement learning across general, agentic, and coding domains and multiple reasoning-effort levels, enabling compositional generalization and robust long-horizon execution. At 2.8T scale, Kimi K3 is supported by infrastructure advances in multiple areas: algorithm-system co-design for KDA, perfectly balanced expert-parallel training with efficient memory management, million-token agentic RL with persistent rollout and sandbox states, and deployment innovations. Extensive evaluations show that Kimi K3 achieves frontier-level performance across long-horizon coding, agentic, knowledge, reasoning, and vision tasks. While its overall performance still trails the most powerful proprietary models, namely Claude Fable 5 and GPT-5.6 Sol, Kimi K3 consistently outperforms other open and proprietary models evaluated in our suite. We release the full Kimi K3 model weights to facilitate future research and accelerate the broader deployment and adoption of frontier intelligence.

cs.CL↗

arXiv2026

An Efficient On-the-Fly Nonadiabatic Coupling Framework Integrated into CP2K

Nonadiabatic molecular dynamics (NAMD) is widely used to describe hot electron relaxation and nonradiative recombination processes, but high computational costs limit its application to large supercells. Here, we implement a nonadiabatic coupling (NAC) module directly into CP2K, enabling on-the-fly NAC calculation during ab-initio molecular dynamics. Unlike conventional approaches relying on interfaces between external NAMD programs and electronic structure codes, this integration streamlines the workflow. Combined with CP2K's inherent Gaussian and Plane Waves (GPW) method, it achieves efficient NAC calculations for large-scale systems. To ensure numerical stability, a phase correction scheme is introduced to remove inconsistencies of wavefunction phases. Benchmark calculations of hot electron relaxation in crystalline pentacene agree with previous studies, while the results of simulations with and without phase correction show significant difference, confirming the necessity of the phase correction. Simulations on larger pentacene supercells further demonstrate the capability of this implementation for large-scale NAMD simulations.

physics.chem-ph↗

arXiv2026

Stable, Fast, and Accurate Kohn-Sham Inversion in Gaussian Basis for Open Shell Molecular and Condensed Phase Systems via Density Matrix Penalization

Here we present a density matrix based KS inversion method formulated entirely within a Gaussian basis representation to optimize a KS potential matrix that reproduces a target electron density. Inverse Kohn-Sham (KS) density functional theory (DFT) aims to determine the effective local KS potential that reproduces a target electron density, and is important both for electronic structure analysis and for the development of orbital based correction methods. In finite Gaussian basis implementations, however, conventional inverse KS-DFT approaches such as the Zhao-Morrison-Parr (ZMP) method often become poorly constrained and inefficient, because the real space penalty potential is projected onto a limited number of Gaussian basis matrix elements, which can strongly coarse-grain its spatial variation. In the present method, the density matrix mismatch is defined in a Lowdin orthogonalized basis, which yields a penalty energy invariant under unitary rotations in that basis. The corresponding penalty potential contribution to the KS Hamiltonian is derived analytically in the original nonorthogonal Gaussian basis. Across a wide range of penalty strengths, the self consistent field (SCF) optimization remains robust and efficient for various open shell systems, while progressively tightening the penalty drives the electron density into accurate agreement with the target. Benchmarks on molecules and condensed phase systems show that the method achieves substantially smaller attainable density deviations than the conventional ZMP method. The method provides a fast and accurate route to KS inversion in finite Gaussian basis sets and may also be useful for future orbital based correction schemes.

physics.chem-ph↗

arXiv2026

Kimi K2.5: Visual Agentic Intelligence

We introduce Kimi K2.5, an open-source multimodal agentic model designed to advance general agentic intelligence. K2.5 emphasizes the joint optimization of text and vision so that two modalities enhance each other. This includes a series of techniques such as joint text-vision pre-training, zero-vision SFT, and joint text-vision reinforcement learning. Building on this multimodal foundation, K2.5 introduces Agent Swarm, a self-directed parallel agent orchestration framework that dynamically decomposes complex tasks into heterogeneous sub-problems and executes them concurrently. Extensive evaluations show that Kimi K2.5 achieves state-of-the-art results across various domains including coding, vision, reasoning, and agentic tasks. Agent Swarm also reduces latency by up to $4.5\times$ over single-agent baselines. We release the post-trained Kimi K2.5 model checkpoint to facilitate future research and real-world applications of agentic intelligence.

cs.CL↗

arXiv2025

The Anisotropic Interface Continuum Solvation Model and the Finite-Element Anisotropic Poisson Solver

We propose an anisotropic interfacial continuum solvation (AICS) model to simulate the distinct in-plane and out-of-plane dielectric constants of liquids near solid-liquid interfaces and their spatial variations along the surface normal direction. In low-electron-density regions, each dielectric function in the diagonal components of a dielectric tensor varies monotonically with distance from the solid surface along the surface normal; in high-electron-density regions near the surface, each dielectric function adopts the electron-density-based formulation proposed by Andreussi et al. (J. Chem. Phys. 136, 064102 (2012)) The resulting dielectric tensor is continuously differentiable with respect to both electron density and spatial coordinates. We derived analytical expressions for electrostatic contributions to the KS potential and forces, and implemented AICS, including these analytical derivatives, into CP2K. To solve the anisotropic Poisson equations, we developed a parallel finite-element anisotropic Poisson solver (FEAPS) based on the FEniCSx platform and its interface with CP2K. Analytical forces were validated against finite-difference calculations, while electrostatic potentials computed under vacuum and isotropic solvent conditions using AICS and FEAPS were benchmarked against standard vacuum DFT and SCCS results, respectively. In the anisotropic solvent environment characterized by the enhanced in-plane and reduced out-of-plane dielectric functions near the Ag(111) surface, we calculated the resulting work functions and electrostatic potentials, and optimized the adsorption geometry for OH. Compared to the isotropic case, we observed more pronounced work function shifts and spatially modulated electrostatic profiles across different charge states. Our results also showed that OH tilted more towards the plane parallel to the surface under the anisotropic dielectric conditions.

physics.chem-ph↗

arXiv2024

BabelBench: An Omni Benchmark for Code-Driven Analysis of Multimodal and Multistructured Data

Large language models (LLMs) have become increasingly pivotal across various domains, especially in handling complex data types. This includes structured data processing, as exemplified by ChartQA and ChatGPT-Ada, and multimodal unstructured data processing as seen in Visual Question Answering (VQA). These areas have attracted significant attention from both industry and academia. Despite this, there remains a lack of unified evaluation methodologies for these diverse data handling scenarios. In response, we introduce BabelBench, an innovative benchmark framework that evaluates the proficiency of LLMs in managing multimodal multistructured data with code execution. BabelBench incorporates a dataset comprising 247 meticulously curated problems that challenge the models with tasks in perception, commonsense reasoning, logical reasoning, and so on. Besides the basic capabilities of multimodal understanding, structured data processing as well as code generation, these tasks demand advanced capabilities in exploration, planning, reasoning and debugging. Our experimental findings on BabelBench indicate that even cutting-edge models like ChatGPT 4 exhibit substantial room for improvement. The insights derived from our comprehensive analysis offer valuable guidance for future research within the community. The benchmark data can be found at https://github.com/FFD8FFE/babelbench.

cs.AI↗

arXiv2024

Functional Analytic Derivation and CP2K Implementation of the SCCS Model Based on the Solvent-Aware Interface

In the self-consistent continuum solvation (SCCS) approach ($\textit{J. Chem. Phys.}$ 136, 064102 (2012)), the analytical expressions of the local solute-solvent interface functions determine the interface function and dielectric function values at a given real space position based solely on the electron density at that position, completely disregarding the surrounding electron density distribution. Therefore, the low electron density areas inside the solute will be identified by the algorithm as regions where implicit solvent exists, resulting in the emergence of non-physical implicit solvent regions within the solute and even potentially leading to the divergence catastrophe of Kohn-Sham SCF calculations. We present a new and efficient SCCS implementation based on the solvent-aware interface ($\textit{J. Chem. Theory Comput.}$ 15, 3, 1996-2009 (2019)) which addresses this issue by utilizing a solute-solvent interface function based on convolution of electron density in the CP2K software package, which is based on the mixed Gaussian and plane waves (GPW) approach. Starting with the foundational formulas of SCCS, we have rigorously and meticulously derived the contributions of the newly defined electrostatic energy to the Kohn-Sham potential and the analytical forces. This comprehensive derivation utilizes the updated versions of the solute-solvent interface function and the dielectric function, tailored to align with the specifics of the GPW implementation. Our implementation has been tested to successfully eliminate non-physical implicit solvent regions within the solute and achieve good SCF convergence, as demonstrated by test results for both bulk and surface models, namely liquid $H_2O$, titanium dioxide, and platinum.

physics.chem-ph↗

arXiv2024

Minimum tracking linear response Hubbard and Hund corrected Density Functional Theory in CP2K

We present the implementation of the Hubbard ($U$) and Hund ($J$) corrected Density Functional Theory (DFT+$U$+$J$) functionality in the Quickstep program, which is part of the CP2K suite. The tensorial and L\"owdin subspace representations are implemented and compared. Full analytical DFT+$U$+$J$ forces are implemented and benchmarked for the tensorial and L\"owdin representations. We also present the implementation of the recently proposed minimum-tracking linear-response method that enables the $U$ and $J$ parameters to be calculated on first principles basis without reference to the Kohn-Sham eigensystem. These implementations are benchmarked against recent results for different materials properties including DFT+$U$ band gap opening in NiO, the relative stability of various polaron distributions in TiO$_2$, the dependence of the calculated TiO$_2$ band gap on +$J$ corrections, and, finally, the role of the +$U$ and +$J$ corrections for the computed properties of a series of the hexahydrated transition metals. Our implementation provides results consistent with those already reported in the literature from comparable methods. We conclude the contribution with tests on the influence of the L\"owdin orthonormalization on the occupancies, calculated parameters, and derived properties.

cond-mat.str-el↗

arXiv2024

An Expert is Worth One Token: Synergizing Multiple Expert LLMs as Generalist via Expert Token Routing

We present Expert-Token-Routing, a unified generalist framework that facilitates seamless integration of multiple expert LLMs. Our framework represents expert LLMs as special expert tokens within the vocabulary of a meta LLM. The meta LLM can route to an expert LLM like generating new tokens. Expert-Token-Routing not only supports learning the implicit expertise of expert LLMs from existing instruction dataset but also allows for dynamic extension of new expert LLMs in a plug-and-play manner. It also conceals the detailed collaboration process from the user's perspective, facilitating interaction as though it were a singular LLM. Our framework outperforms various existing multi-LLM collaboration paradigms across benchmarks that incorporate six diverse expert domains, demonstrating effectiveness and robustness in building generalist LLM system via synergizing multiple expert LLMs.

cs.CL↗

arXiv2024

InfiAgent-DABench: Evaluating Agents on Data Analysis Tasks

In this paper, we introduce InfiAgent-DABench, the first benchmark specifically designed to evaluate LLM-based agents on data analysis tasks. These tasks require agents to end-to-end solving complex tasks by interacting with an execution environment. This benchmark contains DAEval, a dataset consisting of 257 data analysis questions derived from 52 CSV files, and an agent framework which incorporates LLMs to serve as data analysis agents for both serving and evaluation. Since data analysis questions are often open-ended and hard to evaluate without human supervision, we adopt a format-prompting technique to convert each question into a closed-form format so that they can be automatically evaluated. Our extensive benchmarking of 34 LLMs uncovers the current challenges encountered in data analysis tasks. In addition, building on top of our agent framework, we develop a specialized agent, DAAgent, which surpasses GPT-3.5 by 3.9% on DABench. Evaluation datasets and toolkits for InfiAgent-DABench are released at https://github.com/InfiAgent/InfiAgent .

cs.CL↗

arXiv2024

Can GNN be Good Adapter for LLMs?

Recently, large language models (LLMs) have demonstrated superior capabilities in understanding and zero-shot learning on textual data, promising significant advances for many text-related domains. In the graph domain, various real-world scenarios also involve textual data, where tasks and node features can be described by text. These text-attributed graphs (TAGs) have broad applications in social media, recommendation systems, etc. Thus, this paper explores how to utilize LLMs to model TAGs. Previous methods for TAG modeling are based on million-scale LMs. When scaled up to billion-scale LLMs, they face huge challenges in computational costs. Additionally, they also ignore the zero-shot inference capabilities of LLMs. Therefore, we propose GraphAdapter, which uses a graph neural network (GNN) as an efficient adapter in collaboration with LLMs to tackle TAGs. In terms of efficiency, the GNN adapter introduces only a few trainable parameters and can be trained with low computation costs. The entire framework is trained using auto-regression on node text (next token prediction). Once trained, GraphAdapter can be seamlessly fine-tuned with task-specific prompts for various downstream tasks. Through extensive experiments across multiple real-world TAGs, GraphAdapter based on Llama 2 gains an average improvement of approximately 5\% in terms of node classification. Furthermore, GraphAdapter can also adapt to other language models, including RoBERTa, GPT-2. The promising results demonstrate that GNNs can serve as effective adapters for LLMs in TAG modeling.

cs.CL↗

arXiv2024

Enhancing Cross-domain Link Prediction via Evolution Process Modeling

This work proposes DyExpert, a dynamic graph model for cross-domain link prediction. It can explicitly model historical evolving processes to learn the evolution pattern of a specific downstream graph and subsequently make pattern-specific link predictions. DyExpert adopts a decode-only transformer and is capable of efficiently parallel training and inference by \textit{conditioned link generation} that integrates both evolution modeling and link prediction. DyExpert is trained by extensive dynamic graphs across diverse domains, comprising 6M dynamic edges. Extensive experiments on eight untrained graphs demonstrate that DyExpert achieves state-of-the-art performance in cross-domain link prediction. Compared to the advanced baseline under the same setting, DyExpert achieves an average of 11.40% improvement Average Precision across eight graphs. More impressive, it surpasses the fully supervised performance of 8 advanced baselines on 6 untrained graphs.

cs.LG↗

arXiv2023

GraphLLM: Boosting Graph Reasoning Ability of Large Language Model

The advancement of Large Language Models (LLMs) has remarkably pushed the boundaries towards artificial general intelligence (AGI), with their exceptional ability on understanding diverse types of information, including but not limited to images and audio. Despite this progress, a critical gap remains in empowering LLMs to proficiently understand and reason on graph data. Recent studies underscore LLMs' underwhelming performance on fundamental graph reasoning tasks. In this paper, we endeavor to unearth the obstacles that impede LLMs in graph reasoning, pinpointing the common practice of converting graphs into natural language descriptions (Graph2Text) as a fundamental bottleneck. To overcome this impediment, we introduce GraphLLM, a pioneering end-to-end approach that synergistically integrates graph learning models with LLMs. This synergy equips LLMs with the ability to proficiently interpret and reason on graph data, harnessing the superior expressive power of graph learning models. Our empirical evaluations across four fundamental graph reasoning tasks validate the effectiveness of GraphLLM. The results exhibit a substantial average accuracy enhancement of 54.44%, alongside a noteworthy context reduction of 96.45% across various graph reasoning tasks.

cs.CL↗