arXiv subjects

Wen Liu

Publications and source records attributed to Wen Liu.

At least 19 recordsLinked to original sources

arXiv2026

Token-Operations-Oriented Inference Optimization Techniques for Large Models

Large model inference optimization serves as a key foundation for supporting the scalable, low-cost, and highly stable operation of large model services. Centered on token-oriented inference optimization technology, this paper proposes for the first time a four-layer technical architecture consisting of Multi-model Fusion, Model Optimization, Compute-Model Fusion, and Compute-Network-Model Fusion. It systematically reviews the key technologies and current industry status across these four levels and analyzes the application value of related technologies in real-world business scenarios. This paper provides a practical technical path for reducing token production costs, improving token service efficiency, ensuring the stability of token supply, and driving the transition of large model services from being merely callable to being operable.

cs.SE↗

arXiv2026

Reconstruction of fast-rotating neutron star observables with the neural network

Rotation can significantly affect neutron-star (NS) properties, but accurate modeling of rapidly rotating NSs requires solving a two-dimensional, axially symmetric system, making traditional calculations too expensive for inference analyses that demand a large amount of model evaluations. We develop a causal convolutional neural networks that preserve the chronological-like dependence of NS properties on the equation of state (EoS) and rapidly reconstruct observables for static, Keplerian, and rotating configurations. Using \texttt{RNS}, we generate a dataset of NS observables and use it to train our networks. We validate our networks with three representative EoS (SFHo, SLy4, and DD2) and find that the they accurately reproduce the \texttt{RNS} results. The trained networks evaluate NS configurations for a single EoS in $\sim 50$ms, providing a substantial speedup over typical \texttt{RNS} runtimes of $\sim 30$ min and enabling efficient inference analyses involving rapidly rotating NSs.

astro-ph.HE↗

arXiv2026

OMD-GraphRAG: Enhancing GraphRAG with Ontology-Guided Extraction, Multi-Dimensional Clustering and Dual-Channel Fusion

Retrieval-Augmented Generation (RAG) systems face significant challenges in complex reasoning, multi-hop queries, and domain-specific QA. While existing GraphRAG frameworks have made progress in structural knowledge organization, they still have limitations in knowledge extraction precision, community report integrity, and retrieval performance. This paper proposes OMD-GraphRAG, an enhanced framework built upon open-source GraphRAG. The framework introduces three core innovations: (1) Ontology-Guided Knowledge Extraction that uses predefined Schema to guide LLMs in accurately identifying domain-specific entities and relations; (2) Multi-Dimensional Community Clustering Strategy that improves community completeness through alignment completion, attribute-based clustering, and multi-hop relationship clustering; (3) Dual-Channel Graph Retrieval Fusion that balances QA accuracy and performance through hybrid graph and community retrieval. Evaluation results on MultiHop-RAG benchmark show that OMD-GraphRAG outperforms mainstream open source solutions (e.g., LightRAG) in comprehensive F1 scores, particularly in inference and temporal queries.

cs.AI↗

arXiv2026

RTD-Guard: A Black-Box Textual Adversarial Detection Framework via Replacement Token Detection

Textual adversarial attacks pose a serious security threat to Natural Language Processing (NLP) systems by introducing imperceptible perturbations that mislead deep learning models. While adversarial example detection offers a lightweight alternative to robust training, existing methods typically rely on prior knowledge of attacks, white-box access to the victim model, or numerous queries, which severely limits their practical deployment. This paper introduces RTD-Guard, a novel black-box framework for detecting textual adversarial examples. Our key insight is that word-substitution perturbations in adversarial attacks closely resemble the "replaced tokens" that a Replaced Token Detection (RTD) discriminator is pre-trained to identify. Leveraging this, RTD-Guard employs an off-the-shelf RTD discriminator-without fine-tuning-to localize suspicious tokens, masks them, and detects adversarial examples by observing the prediction confidence shift of the victim model before and after intervention. The entire process requires no adversarial data, model tuning, or internal model access, and uses only two black-box queries. Comprehensive experiments on multiple benchmark datasets demonstrate that RTD-Guard effectively detects adversarial texts generated by diverse state-of-the-art attack methods. It surpasses existing detection baselines across multiple metrics, offering a highly efficient, practical, and resource-light defense mechanism-particularly suited for real-world deployment in resource-constrained or privacy-sensitive environments.

cs.CL↗

arXiv2026

Pointer-CAD: Unifying B-Rep and Command Sequences via Pointer-based Edges & Faces Selection

Constructing computer-aided design (CAD) models is labor-intensive but essential for engineering and manufacturing. Recent advances in Large Language Models (LLMs) have inspired the LLM-based CAD generation by representing CAD as command sequences. But these methods struggle in practical scenarios because command sequence representation does not support entity selection (e.g. faces or edges), limiting its ability to support complex editing operations such as chamfer or fillet. Further, the discretization of a continuous variable during sketch and extrude operations may result in topological errors. To address these limitations, we present Pointer-CAD, a novel LLM-based CAD generation framework that leverages a pointer-based command sequence representation to explicitly incorporate the geometric information of B-rep models into sequential modeling. In particular, Pointer-CAD decomposes CAD model generation into steps, conditioning the generation of each subsequent step on both the textual description and the B-rep generated from previous steps. Whenever an operation requires the selection of a specific geometric entity, the LLM predicts a Pointer that selects the most feature-consistent candidate from the available set. Such a selection operation also reduces the quantization error in the command sequence-based representation. To support the training of Pointer-CAD, we develop a data annotation pipeline that produces expert-level natural language descriptions and apply it to build a dataset of approximately 575K CAD models. Extensive experimental results demonstrate that Pointer-CAD effectively supports the generation of complex geometric structures and reduces segmentation error to an extremely low level, achieving a significant improvement over prior command sequence methods, thereby significantly mitigating the topological inaccuracies introduced by quantization error.

cs.CV↗

arXiv2025

Probing Anharmonic Lattice Dynamics and Thermal Transport in Layered Perovskite LiYTiO4 Anode

Layered perovskite lithium yttrium titanate ($\rm LiYTiO_4$) has recently emerged as a promising low-potential, ultrahigh-rate intercalation-type anode material for lithium-ion batteries; however, its lattice dynamics and thermal transport properties remain poorly understood, limiting a complete evaluation of its practical potential. Here, we combine experimental measurements with theoretical modeling to systematically investigate the anharmonic lattice dynamics and heat transport in $\rm LiYTiO_4$. We employ a neural evolution potential (NEP)-based framework that integrates the temperature-dependent effective potential method with the Wigner thermal transport (WTT) formalism, explicitly including both diagonal and off-diagonal terms of the heat-flux operator. Zero-temperature phonon calculations reveal dynamical instabilities associated with $\rm TiO_6$ octahedral rotation, which are stabilized at finite temperatures through anharmonic renormalization. Using the WTT approach with contributions from phonon propagation and coherence contributions, we predict a room-temperature lattice thermal conductivity ($\kappa_{\rm L}$) of 3.8 $\rm Wm^{-1}K^{-1}$ averaged over all crystal orientations, in close agreement with the measured value of 3.2 \pm 0.08 $\rm Wm^{-1}K^{-1}$ for polycrystalline samples. To further examine the possible influence of ionic motion on high-temperature thermal transport, we compute $\kappa_{\rm L}$ using a Green-Kubo equilibrium molecular dynamics approach based on the same NEP, which yields consistent results with both experiment and WTT predictions, confirming the negligible role of Li-ion mobility in heat conduction. Our study not only identifies the ultralow thermal conductivity of $\rm LiYTiO_4$ as a key limitation for its practical application but also establishes a reliable computational framework for studying thermal properties in battery materials.

cond-mat.mtrl-sci↗

arXiv2025

JoyAgent-JDGenie: Technical Report on the GAIA

Large Language Models are increasingly deployed as autonomous agents for complex real-world tasks, yet existing systems often focus on isolated improvements without a unifying design for robustness and adaptability. We propose a generalist agent architecture that integrates three core components: a collective multi-agent framework combining planning and execution agents with critic model voting, a hierarchical memory system spanning working, semantic, and procedural layers, and a refined tool suite for search, code execution, and multimodal parsing. Evaluated on a comprehensive benchmark, our framework consistently outperforms open-source baselines and approaches the performance of proprietary systems. These results demonstrate the importance of system-level integration and highlight a path toward scalable, resilient, and adaptive AI assistants capable of operating across diverse domains and tasks.

cs.CL↗

arXiv2025

DA-Mamba: Dialogue-aware selective state-space model for multimodal engagement estimation

Human engagement estimation in conversational scenarios is essential for applications such as adaptive tutoring, remote healthcare assessment, and socially aware human--computer interaction. Engagement is a dynamic, multimodal signal conveyed by facial expressions, speech, gestures, and behavioral cues over time. In this work we introduce DA-Mamba, a dialogue-aware multimodal architecture that replaces attention-heavy dialogue encoders with Mamba-based selective state-space processing to achieve linear time and memory complexity while retaining expressive cross-modal reasoning. We design a Mamba dialogue-aware selective state-space model composed of three core modules: a Dialogue-Aware Encoder, and two Mamba-based fusion mechanisms: Modality-Group Fusion and Partner-Group Fusion, these modules achieve expressive dialogue understanding. Extensive experiments on three standard benchmarks (NoXi, NoXi-Add, and MPIIGI) show that DA-Mamba surpasses prior state-of-the-art (SOTA) methods in concordance correlation coefficient (CCC), while reducing training time and peak memory; these gains enable processing much longer sequences and facilitate real-time deployment in resource-constrained, multi-party conversational settings. The source code will be available at: https://github.com/kksssssss-ssda/MMEA.

cs.AI↗

arXiv2025

P/D-Device: Disaggregated Large Language Model between Cloud and Devices

Serving disaggregated large language models has been widely adopted in industrial practice for enhanced performance. However, too many tokens generated in decoding phase, i.e., occupying the resources for a long time, essentially hamper the cloud from achieving a higher throughput. Meanwhile, due to limited on-device resources, the time to first token (TTFT), i.e., the latency of prefill phase, increases dramatically with the growth on prompt length. In order to concur with such a bottleneck on resources, i.e., long occupation in cloud and limited on-device computing capacity, we propose to separate large language model between cloud and devices. That is, the cloud helps a portion of the content for each device, only in its prefill phase. Specifically, after receiving the first token from the cloud, decoupling with its own prefill, the device responds to the user immediately for a lower TTFT. Then, the following tokens from cloud are presented via a speed controller for smoothed TPOT (the time per output token), until the device catches up with the progress. On-device prefill is then amortized using received tokens while the resource usage in cloud is controlled. Moreover, during cloud prefill, the prompt can be refined, using those intermediate data already generated, to further speed up on-device inference. We implement such a scheme P/D-Device, and confirm its superiority over other alternatives. We further propose an algorithm to decide the best settings. Real-trace experiments show that TTFT decreases at least 60%, maximum TPOT is about tens of milliseconds, and cloud throughput increases by up to 15x.

cs.DC↗

arXiv2025

Quantum Circuit Optimization Based on Dynamic Grouping and ZX-Calculus for Reducing 2-Qubit Gate Count

In the noisy intermediate-scale quantum (NISQ) era, two-qubit gates in quantum circuits are more susceptible to noise than single-qubit gates. Therefore, reducing the number of two-qubit gates is crucial for improving circuit efficiency and reliability. As quantum circuits scale up, the optimization search space becomes increasingly complex, leading to challenges such as low efficiency and suboptimal solutions. To address these issues, this paper proposes a quantum circuit optimization approach based on dynamic grouping and ZX-calculus. First, a random strategy-based dynamic grouping method partitions the circuit into multiple subcircuits. Second, a ZX-calculus guided k-step lookahead search performs equivalent subcircuit filtering to minimize two-qubit gate counts. Third, a delay-aware placement method optimizes the recombined circuit to reduce the overall gate count. Finally, simulated annealing iteratively updates the grouping strategy to achieve an optimized two-qubit gate count. Experimental results on benchmark datasets demonstrate the effectiveness and superiority of the proposed method in reducing two-qubit gates. Compared to the original circuits, the approach achieves an average reduction of 18% in two-qubit gates. It outperforms classical methods with up to 25% reduction, especially on gf circuits, and shows a 4% average improvement over heuristic ZX-calculus-based methods, validating its efficiency.

quant-ph↗

arXiv2025

Efficient and High-Accuracy Ray Tracing in Discretized Ionospheric Models

High-frequency (HF) ray tracing in complex ionospheric media generally involves a fundamental trade-off between path accuracy and computational efficiency, which directly affects practical applications such as over-the-horizon radar, ionospheric monitoring, and HF skywave communication systems. This paper presents RTM-GD, a ray-tracing framework that combines Hamiltonian ray integration with a continuously differentiable Galerkin--Difference (GD) interpolation strategy. Under discretized ionospheric grid conditions, the electron density is reconstructed as a $C^1$-continuous function within each grid cell, yielding an everywhere differentiable electron-density field for stable numerical integration and improved propagation-path accuracy. Numerical simulations and validations using measured HF oblique sounding data are conducted under diverse conditions, including different ionospheric states, low- and high-elevation angles, multiple operating frequencies, and both ordinary (O) and extraordinary (X) wave modes. Results show that RTM-GD consistently achieves sub-kilometer RMSE in both group-path and ground-distance metrics and sub-0.01-degree azimuth deviation relative to Richardson extrapolation, while reducing computational time by 98\%. Compared with Catmull--Rom interpolation, RTM-GD reduces the RMSEs of ray parameters by approximately one order of magnitude with less than 4\% additional computational cost. Measured-data validation based on ionogram synthesis further shows that the mean relative group-path error remains within 7\%, confirming reliable reproduction of practical HF oblique propagation characteristics. Overall, RTM-GD provides an accurate and computationally efficient framework for HF ray tracing in discretized ionospheric environments.

physics.ao-ph↗

arXiv2025

Enhancing Subject-Independent Accuracy in fNIRS-based Brain-Computer Interfaces with Optimized Channel Selection

Achieving high subject-independent accuracy in functional near-infrared spectroscopy (fNIRS)-based brain-computer interfaces (BCIs) remains a challenge, particularly when minimizing the number of channels. This study proposes a novel feature extraction scheme and a Pearson correlation-based channel selection algorithm to enhance classification accuracy while reducing hardware complexity. Using an open-access fNIRS dataset, our method improved average accuracy by 28.09% compared to existing approaches, achieving a peak subject-independent accuracy of 95.98% with only two channels. These results demonstrate the potential of our optimized feature extraction and channel selection methods for developing efficient, subject-independent fNIRS-based BCI systems.

cs.HC↗

arXiv2025

Quantifying the Capability Boundary of DeepSeek Models: An Application-Driven Performance Analysis

DeepSeek-R1, known for its low training cost and exceptional reasoning capabilities, has achieved state-of-the-art performance on various benchmarks. However, detailed evaluations for DeepSeek Series models from the perspective of real-world applications are lacking, making it challenging for users to select the most suitable DeepSeek models for their specific needs. To address this gap, we presents the first comprehensive evaluation of the DeepSeek and its related models (including DeepSeek-V3, DeepSeek-R1, DeepSeek-R1-Distill-Qwen series, DeepSeek-R1-Distill-Llama series, their corresponding 4-bit quantized models, and the reasoning model QwQ-32B) using our enhanced A-Eval benchmark, A-Eval-2.0. Our systematic analysis reveals several key insights: (1) Given identical model architectures and training data, larger parameter models demonstrate superior performance, aligning with the scaling law. However, smaller models may achieve enhanced capabilities when employing optimized training strategies and higher-quality data; (2) Reasoning-enhanced model show significant performance gains in logical reasoning tasks but may underperform in text understanding and generation tasks; (3) As the data difficulty increases, distillation or reasoning enhancements yield higher performance gains for the models. Interestingly, reasoning enhancements can even have a negative impact on simpler problems; (4) Quantization impacts different capabilities unevenly, with significant drop on logical reasoning and minimal impact on text generation. Based on these results and findings, we design an model selection handbook enabling users to select the most cost-effective models without efforts.

cs.AI↗

arXiv2025

Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling

In this work, we introduce Janus-Pro, an advanced version of the previous work Janus. Specifically, Janus-Pro incorporates (1) an optimized training strategy, (2) expanded training data, and (3) scaling to larger model size. With these improvements, Janus-Pro achieves significant advancements in both multimodal understanding and text-to-image instruction-following capabilities, while also enhancing the stability of text-to-image generation. We hope this work will inspire further exploration in the field. Code and models are publicly available.

cs.AI↗

arXiv2024

DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding

We present DeepSeek-VL2, an advanced series of large Mixture-of-Experts (MoE) Vision-Language Models that significantly improves upon its predecessor, DeepSeek-VL, through two key major upgrades. For the vision component, we incorporate a dynamic tiling vision encoding strategy designed for processing high-resolution images with different aspect ratios. For the language component, we leverage DeepSeekMoE models with the Multi-head Latent Attention mechanism, which compresses Key-Value cache into latent vectors, to enable efficient inference and high throughput. Trained on an improved vision-language dataset, DeepSeek-VL2 demonstrates superior capabilities across various tasks, including but not limited to visual question answering, optical character recognition, document/table/chart understanding, and visual grounding. Our model series is composed of three variants: DeepSeek-VL2-Tiny, DeepSeek-VL2-Small and DeepSeek-VL2, with 1.0B, 2.8B and 4.5B activated parameters respectively. DeepSeek-VL2 achieves competitive or state-of-the-art performance with similar or fewer activated parameters compared to existing open-source dense and MoE-based models. Codes and pre-trained models are publicly accessible at https://github.com/deepseek-ai/DeepSeek-VL2.

cs.CV↗

arXiv2024

JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generation

We present JanusFlow, a powerful framework that unifies image understanding and generation in a single model. JanusFlow introduces a minimalist architecture that integrates autoregressive language models with rectified flow, a state-of-the-art method in generative modeling. Our key finding demonstrates that rectified flow can be straightforwardly trained within the large language model framework, eliminating the need for complex architectural modifications. To further improve the performance of our unified model, we adopt two key strategies: (i) decoupling the understanding and generation encoders, and (ii) aligning their representations during unified training. Extensive experiments show that JanusFlow achieves comparable or superior performance to specialized models in their respective domains, while significantly outperforming existing unified approaches across standard benchmarks. This work represents a step toward more efficient and versatile vision-language models.

cs.CV↗

arXiv2024

CAD-MLLM: Unifying Multimodality-Conditioned CAD Generation With MLLM

This paper aims to design a unified Computer-Aided Design (CAD) generation system that can easily generate CAD models based on the user's inputs in the form of textual description, images, point clouds, or even a combination of them. Towards this goal, we introduce the CAD-MLLM, the first system capable of generating parametric CAD models conditioned on the multimodal input. Specifically, within the CAD-MLLM framework, we leverage the command sequences of CAD models and then employ advanced large language models (LLMs) to align the feature space across these diverse multi-modalities data and CAD models' vectorized representations. To facilitate the model training, we design a comprehensive data construction and annotation pipeline that equips each CAD model with corresponding multimodal data. Our resulting dataset, named Omni-CAD, is the first multimodal CAD dataset that contains textual description, multi-view images, points, and command sequence for each CAD model. It contains approximately 450K instances and their CAD construction sequences. To thoroughly evaluate the quality of our generated CAD models, we go beyond current evaluation metrics that focus on reconstruction quality by introducing additional metrics that assess topology quality and surface enclosure extent. Extensive experimental results demonstrate that CAD-MLLM significantly outperforms existing conditional generative methods and remains highly robust to noises and missing points. The project page and more visualizations can be found at: https://cad-mllm.github.io/

cs.CV↗

arXiv2024

Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation

In this paper, we introduce Janus, an autoregressive framework that unifies multimodal understanding and generation. Prior research often relies on a single visual encoder for both tasks, such as Chameleon. However, due to the differing levels of information granularity required by multimodal understanding and generation, this approach can lead to suboptimal performance, particularly in multimodal understanding. To address this issue, we decouple visual encoding into separate pathways, while still leveraging a single, unified transformer architecture for processing. The decoupling not only alleviates the conflict between the visual encoder's roles in understanding and generation, but also enhances the framework's flexibility. For instance, both the multimodal understanding and generation components can independently select their most suitable encoding methods. Experiments show that Janus surpasses previous unified model and matches or exceeds the performance of task-specific models. The simplicity, high flexibility, and effectiveness of Janus make it a strong candidate for next-generation unified multimodal models.

cs.CV↗