arXiv subjects

Han Xu

Publications and source records attributed to Han Xu.

At least 37 records · Page 2Linked to original sources

arXiv2026

GLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agents

We present GLM-5V-Turbo, a step toward native foundation models for multimodal agents. As foundation models are increasingly deployed in real environments, agentic capability depends not only on language reasoning, but also on the ability to perceive, interpret, and act over heterogeneous contexts such as images, videos, webpages, documents, GUIs. GLM-5V-Turbo is built around this objective: multimodal perception is integrated as a core component of reasoning, planning, tool use, and execution, rather than as an auxiliary interface to a language model. This report summarizes the main improvements behind GLM-5V-Turbo across model design, multimodal training, reinforcement learning, toolchain expansion, and integration with agent frameworks. These developments lead to strong performance in multimodal coding, visual tool use, and framework-based agentic tasks, while preserving competitive text-only coding capability. More importantly, our development process offers practical insights for building multimodal agents, highlighting the central role of multimodal perception, hierarchical optimization, and reliable end-to-end verification.

cs.CV↗

arXiv2026

SpikingBrain: Spiking Brain-inspired Large Models

Mainstream Transformer-based large language models face major efficiency bottlenecks: training computation scales quadratically with sequence length, and inference memory grows linearly, limiting long-context processing. Building large models on non-NVIDIA platforms also poses challenges for stable and efficient training. To address this, we introduce SpikingBrain, a family of brain-inspired models designed for efficient long-context training and inference. SpikingBrain leverages the MetaX GPU cluster and focuses on three aspects: (1) Model Architecture: linear and hybrid-linear attention architectures with adaptive spiking neurons; (2) Algorithmic Optimizations: an efficient, conversion-based training pipeline and a dedicated spike coding framework; (3) System Engineering: customized training frameworks, operator libraries, and parallelism strategies tailored to MetaX hardware. Using these techniques, we develop two models: SpikingBrain-7B, a linear LLM, and SpikingBrain-76B, a hybrid-linear MoE LLM. These models demonstrate the feasibility of large-scale LLM development on non-NVIDIA platforms, and training remains stable for weeks on hundreds of MetaX GPUs with Model FLOPs Utilization at expected levels. SpikingBrain achieves performance comparable to open-source Transformer baselines while using only about 150B tokens for continual pre-training. Our models also significantly improve long-context efficiency and deliver inference with (partially) constant memory and event-driven spiking behavior. For example, SpikingBrain-7B attains over 100x speedup in Time to First Token for 4M-token sequences. Furthermore, the proposed spiking scheme achieves 69.15 percent sparsity, enabling low-power operation. Overall, this work demonstrates the potential of brain-inspired mechanisms to drive the next generation of efficient and scalable large model design.

cs.LG↗

arXiv2026

SpikingBrain2.0: Brain-Inspired Foundation Models for Efficient Long-Context and Cross-Platform Inference

Scaling context length is reshaping large-model development, yet full-attention Transformers suffer from prohibitive computation and inference bottlenecks at long sequences. A key challenge is to design foundation models that maintain performance and long-context efficiency with minimal training overhead. We introduce SpikingBrain2.0 (SpB2.0), a 5B model that advances both architecture and training efficiency of its predecessor. Our contributions are two-fold. (1) Architectural Innovation: We propose Dual-Space Sparse Attention (DSSA), an inter-layer hybrid of Sparse Softmax Attention (MoBA) and Sparse Linear Attention (SSE), achieving an improved performance-efficiency trade-off for long-context modeling. SpB2.0 further supports dual quantization paths: INT8-Spiking coding enables sparse event-driven computation, while FP8 coding accelerates inference on modern GPUs. (2) Enhanced Training Strategy: We develop an optimized Transformer-to-Hybrid (T2H) pipeline with dual conversion paths for LLMs and VLMs using curated open-source data. Empirically, SpB2.0-5B and SpB2.0-VL-5B recover most of the base Transformer (Qwen3-4B) capability with under 7k A100 GPU hours. SpB2.0 achieves a 10.13x TTFT speedup at 4M context and supports over 10M tokens on 8 A100 GPUs under vLLM, where full-attention models exceed memory limits. It also demonstrates strong cross-platform compatibility, enabling FP8 GPU inference (2.52x speedup at 250k) and efficient neuromorphic execution (64.31% sparsity, with 70.6% and 46.5% area and power reduction at 500MHz). Overall, SpikingBrain2.0 provides a practical pathway for lightweight, multimodal, spiking foundation models, highlighting the potential of combining brain-inspired mechanisms with efficient architectures for resource-constrained and edge scenarios.

cs.LG↗

arXiv2026

SpikeMLLM: Spike-based Multimodal Large Language Models via Modality-Specific Temporal Scales and Temporal Compression

Multimodal Large Language Models (MLLMs) have achieved remarkable progress but incur substantial computational overhead and energy consumption during inference, limiting deployment in resource-constrained environments. Spiking Neural Networks (SNNs), with their sparse event-driven computation, offer inherent energy efficiency advantages on neuromorphic hardware, yet extending them to MLLMs faces two key challenges: heterogeneous modalities make uniform spike encoding insufficient, and high-resolution image inputs amplify timestep unfolding overhead. We propose SpikeMLLM, the first spike-based framework for MLLMs, which unifies existing ANN quantization methods in the spiking representation space and incorporates Modality-Specific Temporal Scales (MSTS) guided by Modality Evolution Discrepancy (MED) and Temporally Compressed LIF (TC-LIF) for timestep compression from T=L-1 to T=log2(L)-1. Experiments on four representative MLLMs across diverse multimodal benchmarks show that SpikeMLLM maintains near-lossless performance under aggressive timestep compression (Tv/Tt=3/4), with average gaps of only 0.72% and 1.19% relative to the FP16 baseline on InternVL2-8B and Qwen2VL-72B. We further develop a dedicated RTL accelerator tailored to the spike-driven datapath, observing 9.06x higher throughput and 25.8x better power efficiency relative to an FP16 GPU baseline under a deployment-oriented co-design setting, suggesting the promise of algorithm-hardware co-design for efficient multimodal intelligence.

cs.NE↗

arXiv2026

Spike-driven Large Language Model

Current Large Language Models (LLMs) are primarily based on large-scale dense matrix multiplications. Inspired by the brain's information processing mechanism, we explore the fundamental question: how to effectively integrate the brain's spiking-driven characteristics into LLM inference. Spiking Neural Networks (SNNs) possess spike-driven characteristics, and some works have attempted to combine SNNs with Transformers. However, achieving spike-driven LLMs with billions of parameters, relying solely on sparse additions, remains a challenge in the SNN field. To address the issues of limited representational capacity and sparsity in existing spike encoding schemes at the LLM level, we propose SDLLM, a spike-driven large language model that eliminates dense matrix multiplications through sparse addition operations. Specifically, we use the plug-and-play gamma-SQP two-step spike encoding method to ensure that the quantization process aligns with the model's semantic space, mitigating representation degradation caused by binary spikes. Furthermore, we introduce bidirectional encoding under symmetric quantization and membrane potential clipping mechanisms, leading to spike trains with no or low firing counts dominating, significantly reducing the model's spike firing rate, while halving the number of time steps. Experimental results show that SDLLM not only significantly reduces inference costs but also achieves state-of-the-art task performance under the spike-based paradigm. For example, compared to previous spike-based LLMs, SDLLM reduces energy consumption by 7x and improves accuracy by 4.2%. Our model provides inspiration for the architecture design of the next generation of event-driven neuromorphic chips.

cs.NE↗

arXiv2026

Phase transitions in parametrized quantum circuits

Phase transitions are among the most intriguing phenomena in physical systems, yet their behavior near criticality remain challenging to study using classical algorithms. Parameterized quantum circuits (PQCs) offer a promising approach to investigating such regimes on practical quantum computers. However, in order to use it to probe critical behavior, a PQC itself should be non-trivial and exhibit a phase transition and non-analyticity -- a property that has not yet been clearly identified. In this work, we identify a mechanism for generating non-analyticities intrinsically in PQCs. As a concrete realization, we construct a class of sequential PQCs whose observable expectation value is a non-analytic function of the circuit parameter in the infinite volume limit, showing that the prepared PQC states undergo a phase transition at the non-analytic points. The entanglement and the identified order parameter have distinct behaviors in different phases, revealing a phase diagram of the PQC state. We show that classical simulation of this PQC based on tensor networks and Pauli propagation gets less efficient in the vicinity of the phase transition point, indicating a physically motivated route towards practical quantum advantage using PQCs with phase transitions.

quant-ph↗

arXiv2026

Ultralow and Tunable Thermal Conductivity of Parylene C for Thermal Insulation in Advanced Packaging

Parylene C thin films have significant applications in advanced packaging of microelectronics. Their thermal properties are critical for thermal management of electronic devices. However, a unified understanding of the tunable structure and the corresponding thermal conductivity is still missing. This study investigated parylene C thin films of varying thickness and post-annealing temperatures grown via thermal chemical vapor deposition. The ultralow thermal conductivity of as-deposited parylene C measured by time domain thermoreflectance (TDTR) is 0.10 W/m-K. The thermal conductivity can be tuned by post-annealing. Significant increase in thermal conductivity is observed in the annealed samples (0.18 W/m-K) which induces melting and recrystallization. The results of XRD and polarized Raman spectroscopy show that the enhanced thermal conductivity is due to improved crystalline quality and the change in chain orientations. The measured thermal conductivities of the as-deposited and annealed films are much lower than the values predicted by the Cahill minimum thermal conductivity model, which can be explained by the diffuson-mediated minimum thermal conductivity model. Parylene C is found to possess the lowest thermal conductivity among dense low-k materials. Our work provides guidance for the structural design of ultra-low thermal conductivity polymers and corresponding thermal design of electronics.

cond-mat.mtrl-sci↗

arXiv2026

Simulation of a Heterogeneous Quantum Network

Quantum networks are expected to be heterogeneous systems, combining distinct qubit platforms, photon wavelengths, and device timescales to achieve scalable, multiuser connectivity. Building and iterating on such systems is costly and slow, which motivates hardware-faithful simulations that explore architecture design space and justify implementation decisions. This paper presents a framework for simulating heterogeneous quantum networks based on SeQUeNCe, a discrete-event simulator of quantum networks. We introduce faithful device models for two representative platforms - Ytterbium atoms and superconducting qubits - to implement entanglement generation and swapping protocols for time-bin encoded photons. Using extensive simulations that account for disparate clock rates and quantum frequency conversion and transduction losses/noise, we map the rate-fidelity trade space and identify the dominant bottlenecks unique to heterogeneous systems. The models are open source and extensible, enabling reproducible evaluation of future heterogeneous designs and protocols.

quant-ph↗

arXiv2026

VideoFusion: A Spatio-Temporal Collaborative Network for Multi-modal Video Fusion

Compared to images, videos better reflect real-world acquisition and possess valuable temporal cues. However, existing multi-sensor fusion research predominantly integrates complementary context from multiple images rather than videos due to the scarcity of large-scale multi-sensor video datasets, limiting research in video fusion and the inherent difficulty of jointly modeling spatial and temporal dependencies in a unified framework. To this end, we construct M3SVD, a benchmark dataset with $220$ temporally synchronized and spatially registered infrared-visible videos comprising $153,797$ frames, bridging the data gap. Secondly, we propose VideoFusion, a multi-modal video fusion model that exploits cross-modal complementarity and temporal dynamics to generate spatio-temporally coherent videos from multi-modal inputs. Specifically, 1) a differential reinforcement module is developed for cross-modal information interaction and enhancement, 2) a complete modality-guided fusion strategy is employed to adaptively integrate multi-modal features, and 3) a bi-temporal co-attention mechanism is devised to dynamically aggregate forward-backward temporal contexts to reinforce cross-frame feature representations. Experiments reveal that VideoFusion outperforms existing image-oriented fusion paradigms in sequences, effectively mitigating temporal inconsistency and interference. Project and M3SVD: https://github.com/Linfeng-Tang/VideoFusion.

cs.CV↗

arXiv2026

Dependently-Typed AARA: A Non-Affine Approach for Resource Analysis of Higher-Order Programs

Static resource analysis determines the resource consumption (e.g., time complexity) of a program without executing it. Among the numerous existing approaches for resource analysis, affine type systems have been one dominant approach. However, these affine type systems fall short of deriving precise resource behavior of higher-order programs, particularly in cases that involve partial applications. This article presents λ_\ms{amor}^\ms{na}}, a non-affine AARA-style dependent type system for resource reasoning about higher-order functional programs. The key observation is that the main issue in previous approaches comes from (i) the close coupling of types and resources, and (ii) the conflict between affine and higher-order typing mechanisms. To derive precise resource behavior of higher-order functions, λ_\ms{amor}^\ms{na}} decouples resources from types and follows a non-affine typing mechanism. The non-affine type system of λ_\ms{amor}^\ms{na}} achieves this by using dependent types, which allows expressing type-level potential functions separate from ordinary types. This article formalizes λ_\ms{amor}^\ms{na}}'s syntax and semantics, and proves its soundness, which guarantees the correctness of resource bounds. Several challenging classic and higher-order examples are presented to demonstrate the expressiveness and compositionality of λ_\ms{amor}^\ms{na}}'s reasoning capability.

cs.PL↗

arXiv2026

Coherent Mode Decoupling: A Versatile Framework for High-Throughput Partially Coherent Light Transport

Accurate and efficient wave-optics simulation of partially coherent light transport systems is critical for the design of advanced optical systems, ranging from computational lithography to diffraction-limited storage rings (DLSR). However, traditional approaches based on Coherent Mode Decomposition suffer from high computational costs due to the propagating massive sets of two-dimensional modes. In this paper, we propose the Coherent Mode Decoupling (CMDC) algorithm, a high-throughput computational framework designed to accelerate these simulations by orders of magnitude without compromising physical fidelity. The method factorizes 2D modes into efficient one-dimensional (1D) components, while crucially incorporating a subspace compression strategy to capture non-separable coupling effects. We demonstrated the generality and robustness of this framework in applications ranging from computational lithography to coherent beamlines of DLSR.

physics.optics↗

arXiv2026

Interpretable Probability Estimation with LLMs via Shapley Reconstruction

Large Language Models (LLMs) demonstrate potential to estimate the probability of uncertain events, by leveraging their extensive knowledge and reasoning capabilities. This ability can be applied to support intelligent decision-making across diverse fields, such as financial forecasting and preventive healthcare. However, directly prompting LLMs for probability estimation faces significant challenges: their outputs are often noisy, and the underlying predicting process is opaque. In this paper, we propose PRISM: Probability Reconstruction via Shapley Measures, a framework that brings transparency and precision to LLM-based probability estimation. PRISM decomposes an LLM's prediction by quantifying the marginal contribution of each input factor using Shapley values. These factor-level contributions are then aggregated to reconstruct a calibrated final estimate. In our experiments, we demonstrate PRISM improves predictive accuracy over direct prompting and other baselines, across multiple domains including finance, healthcare, and agriculture. Beyond performance, PRISM provides a transparent prediction pipeline: our case studies visualize how individual factors shape the final estimate, helping build trust in LLM-based decision support systems.

cs.LG↗

arXiv2026

On Discrete Age of Information of Status Updating System With General Packet Arrival Processes

Characterizing Age of Information (AoI) in status updating systems with general arrival and service processes has great significance considering that the interarrival and service time of updates can possibly be arbitrary in a real world. While expressions of average continuous AoI under G/G/1/1 queues have been derived in the paper by Soysal and Ulukus, the discrete case remained unsolved. To address it, this paper gives a fully characterization of probability generation functions (PGF) of discrete AoI under G/G/1/1 settings when preemption is allowed. In the non-preemptive case, this paper gives the expressions of PGF of discrete AoI under G/Geo/1/1 settings, which also extends the former results. The average discrete AoI is derived and discussed based on these new theoretical findings.

eess.SY↗

arXiv2026

Spin waves excited by hard x-ray transient gratings

Recent progress in ultrafast x-ray sources helped establish x-rays as an important tool for probing lattice and magnetic dynamics initiated by femtosecond optical pulses. Here, we explore the potential of ultrashort hard x-ray pulses for driving magnetic dynamics. We use a transient grating technique in which a spatially periodic x-ray excitation pattern gives rise to material excitations at a well-defined wave vector, whose dynamics are monitored via diffraction of an optical probe pulse. The excitation of a ferrimagnetic gadolinium bismuth iron garnet film placed in an external tilted magnetic field by x-rays at the Gd L3 edge results in both magnetic and non-magnetic transient gratings whose contributions to the diffracted signal are separated by polarization analysis. We observe the magnetization precession at both longitudinal acoustic and spin wave frequencies. An analysis with the Landau-Lifshitz-Gilbert equation indicates that the magnetization precession is driven by strain resulting from thermal expansion induced by absorbed x-rays. The results establish x-ray transient gratings as a tool for driving coherent phonons and magnons, with the potential of accessing wave vectors across the entire Brillouin zone.

physics.optics↗

arXiv2025

WaveC2R: Wavelet-Driven Coarse-to-Refined Hierarchical Learning for Radar Retrieval

Satellite-based radar retrieval methods are widely employed to fill coverage gaps in ground-based radar systems, especially in remote areas affected by terrain blockage and limited detection range. Existing methods predominantly rely on overly simplistic spatial-domain architectures constructed from a single data source, limiting their ability to accurately capture complex precipitation patterns and sharply defined meteorological boundaries. To address these limitations, we propose WaveC2R, a novel wavelet-driven coarse-to-refined framework for radar retrieval. WaveC2R integrates complementary multi-source data and leverages frequency-domain decomposition to separately model low-frequency components for capturing precipitation patterns and high-frequency components for delineating sharply defined meteorological boundaries. Specifically, WaveC2R consists of two stages (i)Intensity-Boundary Decoupled Learning, which leverages wavelet decomposition and frequency-specific loss functions to separately optimize low-frequency intensity and high-frequency boundaries; and (ii)Detail-Enhanced Diffusion Refinement, which employs frequency-aware conditional priors and multi-source data to progressively enhance fine-scale precipitation structures while preserving coarse-scale meteorological consistency. Experimental results on the publicly available SEVIR dataset demonstrate that WaveC2R achieves state-of-the-art performance in satellite-based radar retrieval, particularly excelling at preserving high-intensity precipitation features and sharply defined meteorological boundaries.

eess.SP↗

arXiv2025

DT-MPC: Synthesizing Derivation-Free Model Predictive Control from Power Converter Netlists via Physics-Informed Neural Digital Twins

Model Predictive Control (MPC) is a powerful control strategy for power electronics, but it highly relies on manually-derived and topology-specific analytical models, which is labor-intensive and time-consuming in practical designs. To overcome this bottleneck, this paper introduces a Digital-Twin-based MPC (DT-MPC) framework for generic power converters that can systematically translate a high-level circuit into an objective-aware control policy by leveraging a DT as a high-fidelity system model. Furthermore, a physics-informed neural surrogate predictor is proposed to accelerate predictions by DT and enable real-time operation. A gradient-free simplex search optimizer is also introduced to efficiently handle complex multi-objective optimization. The efficacy of the framework has been validated through a cloud-to-edge deployment on a 1500 W dual active bridge converter. Experimental results show that the synthesized predictive model achieves an inference speed over 7 times faster than real time, the DT-MPC controller outperforms several human-designed counterparts, and the overall framework reduces engineering design time by over 95\%, verifying the superiority of DT-MPC on generalized power converters.

eess.SY↗

arXiv2025

Ensemble Privacy Defense for Knowledge-Intensive LLMs against Membership Inference Attacks

Retrieval-Augmented Generation (RAG) and Supervised Finetuning (SFT) have become the predominant paradigms for equipping Large Language Models (LLMs) with external knowledge for diverse, knowledge-intensive tasks. However, while such knowledge injection improves performance, it also exposes new attack surfaces. Membership Inference Attacks (MIAs), which aim to determine whether a given data sample was included in a model's training set, pose serious threats to privacy and trust in sensitive domains. To this end, we first systematically evaluate the vulnerability of RAG- and SFT-based LLMs to various MIAs. Then, to address the privacy risk, we further introduce a novel, model-agnostic defense framework, Ensemble Privacy Defense (EPD), which aggregates and evaluates the outputs of a knowledge-injected LLM, a base LLM, and a dedicated judge model to enhance resistance against MIAs. Comprehensive experiments show that, on average, EPD reduces MIA success by up to 27.8\% for SFT and 526.3\% for RAG compared to inference-time baseline, while maintaining answer quality.

cs.CR↗

arXiv2025

Deep Learning and Machine Learning -- Natural Language Processing: From Theory to Application

With a focus on natural language processing (NLP) and the role of large language models (LLMs), we explore the intersection of machine learning, deep learning, and artificial intelligence. As artificial intelligence continues to revolutionize fields from healthcare to finance, NLP techniques such as tokenization, text classification, and entity recognition are essential for processing and understanding human language. This paper discusses advanced data preprocessing techniques and the use of frameworks like Hugging Face for implementing transformer-based models. Additionally, it highlights challenges such as handling multilingual data, reducing bias, and ensuring model robustness. By addressing key aspects of data processing and model fine-tuning, this work aims to provide insights into deploying effective and ethically sound AI solutions.

cs.CL↗