arXiv subjects

Lingxiao Zhao

Publications and source records attributed to Lingxiao Zhao.

At least 37 records · Page 2Linked to original sources

arXiv2025

Voxtral

We present Voxtral Mini and Voxtral Small, two multimodal audio chat models. Voxtral is trained to comprehend both spoken audio and text documents, achieving state-of-the-art performance across a diverse range of audio benchmarks, while preserving strong text capabilities. Voxtral Small outperforms a number of closed-source models, while being small enough to run locally. A 32K context window enables the model to handle audio files up to 40 minutes in duration and long multi-turn conversations. We also contribute three benchmarks for evaluating speech understanding models on knowledge and trivia. Both Voxtral models are released under Apache 2.0 license.

cs.SD↗

arXiv2025

Magistral

We introduce Magistral, Mistral's first reasoning model and our own scalable reinforcement learning (RL) pipeline. Instead of relying on existing implementations and RL traces distilled from prior models, we follow a ground up approach, relying solely on our own models and infrastructure. Notably, we demonstrate a stack that enabled us to explore the limits of pure RL training of LLMs, present a simple method to force the reasoning language of the model, and show that RL on text data alone maintains most of the initial checkpoint's capabilities. We find that RL on text maintains or improves multimodal understanding, instruction following and function calling. We present Magistral Medium, trained for reasoning on top of Mistral Medium 3 with RL alone, and we open-source Magistral Small (Apache 2.0) which further includes cold-start data from Magistral Medium.

cs.CL↗

arXiv2025

Orientation- and pressure-dependence of the vibrational response of a monolayer crystal on a vicinal diamond surface

We systematically investigate polarization-dependent Raman spectra of a monolayer crystal of WS2 on the (100) and (230) surfaces of diamond. At ambient pressure, identical polarization dependence of the Raman spectra is observed on the different surfaces, independent of the orientation of the monolayer crystal relative to the diamond crystal. However, when monolayer WS2 is compressed to about 4 GPa, an abrupt drop of the intensity of the 2LA mode relative to the A' mode occurs on the (100) surface and the (230) surface with the zigzag direction along the atomic step edges of the (230) surface. In contrast, no such drop is observed when the armchair direction is along or at 15° to the atomic steps of the (230) surface. We also observe a shift of the polarization angle of the intensity maxima of the 2LA and A' modes on (230) surface during compression. These results demonstrate that the atomic steps of a vicinal surface strongly modify the vibrational response under high pressure.

cond-mat.mtrl-sci↗

arXiv2025

Lattice dynamics and spin-phonon coupling in the kagome spin ice HoAgGe

We employ polarization-resolved Raman spectroscopy and first-principles phonon calculations to study the lattice dynamics and spin-phonon coupling in the kagome spin ice compound HoAgGe. Upon cooling, HoAgGe shows transitions from a nonmagnetic state at 300 K to a partially magnetic-ordered phase below T2 = 11.6 K, eventually reaching a fully-magnetic ordered phase below T1 = 7 K. We detect eight of ten Raman-active phonon modes at room temperature, with frequencies consistent with first-principles phonon calculations. We find that the low-energy phonon modes harden upon cooling, soften slightly below T2, and finally harden again below T1, suggesting finite spin-phonon coupling in HoAgGe. Furthermore, one high-energy mode at 185 cm-1 with E' symmetry exhibits a Fano lineshape due to electron-phonon coupling. Our Raman results establish finite spin-phonon coupling and electron-phonon coupling in the magnetic phase of HoAgGe, providing a foundation for future studies of the novel field-induced phases at low temperatures in HoAgGe.

cond-mat.str-el↗

arXiv2025

Chain of Draft: Thinking Faster by Writing Less

Large Language Models (LLMs) have demonstrated remarkable performance in solving complex reasoning tasks through mechanisms like Chain-of-Thought (CoT) prompting, which emphasizes verbose, step-by-step reasoning. However, humans typically employ a more efficient strategy: drafting concise intermediate thoughts that capture only essential information. In this work, we propose Chain of Draft (CoD), a novel paradigm inspired by human cognitive processes, where LLMs generate minimalistic yet informative intermediate reasoning outputs while solving tasks. By reducing verbosity and focusing on critical insights, CoD matches or surpasses CoT in accuracy while using as little as only 7.6% of the tokens, significantly reducing cost and latency across various reasoning tasks. Our code and data are available at https://github.com/sileix/chain-of-draft.

cs.CL↗

arXiv2024

T-REG: Preference Optimization with Token-Level Reward Regularization

Reinforcement learning from human feedback (RLHF) has been crucial in aligning large language models (LLMs) with human values. Traditionally, RLHF involves generating responses to a query and using a reward model to assign a reward to the entire response. However, this approach faces challenges due to its reliance on a single, sparse reward, which makes it challenging for the model to identify which parts of the sequence contribute most significantly to the final reward. Recent methods have attempted to address this limitation by introducing token-level rewards. However, these methods often rely on either a trained credit assignment model or AI annotators, raising concerns about the quality and reliability of the rewards. In this paper, we propose token-level reward regularization (T-REG), a novel approach that leverages both sequence-level and token-level rewards for preference optimization. Harnessing the self-refinement capabilities of LLMs, our method uses contrastive prompting to enable LLMs to self-generate token-level rewards. These self-generated rewards then act as reward regularization, guiding the model to more effectively distribute sequence-level rewards across tokens. This facilitates better token-level credit assignment and enhances alignment performance. Experiments on the instruction following benchmarks, including Alpaca Eval 2 and Arena-Hard, show that our method consistently outperforms baseline methods by up to 3.8% and 4.4%, respectively. We will release the code and models at https://github.com/wzhouad/T-REG.

cs.CL↗

arXiv2024

Pard: Permutation-Invariant Autoregressive Diffusion for Graph Generation

Graph generation has been dominated by autoregressive models due to their simplicity and effectiveness, despite their sensitivity to ordering. Yet diffusion models have garnered increasing attention, as they offer comparable performance while being permutation-invariant. Current graph diffusion models generate graphs in a one-shot fashion, but they require extra features and thousands of denoising steps to achieve optimal performance. We introduce PARD, a Permutation-invariant Auto Regressive Diffusion model that integrates diffusion models with autoregressive methods. PARD harnesses the effectiveness and efficiency of the autoregressive model while maintaining permutation invariance without ordering sensitivity. Specifically, we show that contrary to sets, elements in a graph are not entirely unordered and there is a unique partial order for nodes and edges. With this partial order, PARD generates a graph in a block-by-block, autoregressive fashion, where each block's probability is conditionally modeled by a shared diffusion model with an equivariant network. To ensure efficiency while being expressive, we further propose a higher-order graph transformer, which integrates transformer with PPGN. Like GPT, we extend the higher-order graph transformer to support parallel training of all blocks. Without any extra features, PARD achieves state-of-the-art performance on molecular and non-molecular datasets, and scales to large datasets like MOSES containing 1.9M molecules. Pard is open-sourced at https://github.com/LingxiaoShawn/Pard.

cs.LG↗

arXiv2024

Thermodynamic origin of the phonon Hall effect in a honeycomb antiferromagnet

The underlying mechanism of the thermal Hall effect (THE) generated by phonons in a variety of insulators is yet to be identified. Here, we report on a sizeable thermal Hall conductivity in NiPS$_3$, a van der Waals stack of honeycomb layers with a zigzag antiferromagnetic order below $T_N$ = 155 K. The longitudinal ($κ_{aa}$) and the transverse ($κ_{ab}$) thermal conductivities peak at the same temperature and the thermal Hall angle, at this peak, respects a previously identified bound. The amplitude of $κ_{ab}$ is extremely sensitive to the amplitude of magnetization along the $b$-axis, in contrast to the phonon mean free path, which is not at all. We show that the magnon and acoustic phonon bands cross each other along the $b^\ast$ orientation in the momentum space. The relevance of a thermodynamic property, combined with the irrelevance of the mean free path, points to an intrinsic origin.

cond-mat.str-el↗

arXiv2024

Unified Discrete Diffusion for Categorical Data

Discrete diffusion models have seen a surge of attention with applications on naturally discrete data such as language and graphs. Although discrete-time discrete diffusion has been established for a while, only recently Campbell et al. (2022) introduced the first framework for continuous-time discrete diffusion. However, their training and sampling processes differ significantly from the discrete-time version, necessitating nontrivial approximations for tractability. In this paper, we first present a series of mathematical simplifications of the variational lower bound that enable more accurate and easy-to-optimize training for discrete diffusion. In addition, we derive a simple formulation for backward denoising that enables exact and accelerated sampling, and importantly, an elegant unification of discrete-time and continuous-time discrete diffusion. Thanks to simpler analytical formulations, both forward and now also backward probabilities can flexibly accommodate any noise distribution, including different noise distributions for multi-element objects. Experiments show that our proposed USD3 (for Unified Simplified Discrete Denoising Diffusion) outperform all SOTA baselines on established datasets. We open-source our unified code at https://github.com/LingxiaoShawn/USD3.

cs.LG↗

arXiv2024

Local excitation of kagome spin ice magnetism in HoAgGe seen by scanning tunneling microscopy

The kagome spin ice can host frustrated magnetic excitations by flipping its local spin. Under an inelastic tunneling condition, the tip in a scanning tunneling microscope can flip the local spin, and we apply this technique to kagome metal HoAgGe with a long-range ordered spin ice ground state. Away from defects, we discover a pair of pronounced dips in the local tunneling spectrum at symmetrical bias voltages with negative intensity values, serving as a striking inelastic tunneling signal. This signal disappears above the spin ice formation temperature and has a dependence on the magnetic fields, demonstrating its intimate relation with the spin ice magnetism. We provide a two-level spin-flip model to explain the tunneling dips considering the spin ice magnetism under spin-orbit coupling. Our results uncover a local emergent excitation of spin ice magnetism in a kagome metal, suggesting that local electrical field induced spin flip climbs over a barrier caused by spin-orbital locking.

cond-mat.str-el↗

arXiv2024

Granular Ta-Te nanowire superconductivity violating the Pauli limit

Strategies to achieve higher upper-critical-field superconductors (μ0Hc2(0)) are of great interest for both fundamental science and practical applications. While reducing the thickness of two-dimensional (2D) materials to a few layers significantly enhances μ0Hc2(0) with accompanied potential unconventional pairing mechanisms, further dimensional reduction to 1D compounds rarely exceeds the expected Pauli limit. Here, we report the discovery of a 1D granular Ta-Te nanowire that becomes superconducting under high pressure, with a maximum critical temperature (Tc) of 5.1 K. Remarkably, the μ0Hc2(0) reaches 16 T, which is twice the Pauli limit, setting a record of μ0Hc2 (0) in all the reported 1D superconductors. Our work demonstrates that the Ta-Te nanowire not only is a potential candidate for applications in high magnetic fields, but also provides an ideal platform for further investigations of the mechanisms between nanowires and large μ0Hc2(0).

cond-mat.supr-con↗

arXiv2024

Improving Multilingual Instruction Finetuning via Linguistically Natural and Diverse Datasets

Advancements in Large Language Models (LLMs) have significantly enhanced instruction-following capabilities. However, most Instruction Fine-Tuning (IFT) datasets are predominantly in English, limiting model performance in other languages. Traditional methods for creating multilingual IFT datasets such as translating existing English IFT datasets or converting existing NLP datasets into IFT datasets by templating, struggle to capture linguistic nuances and ensure prompt (instruction) diversity. To address this issue, we propose a novel method for collecting multilingual IFT datasets that preserves linguistic naturalness and ensures prompt diversity. This approach leverages English-focused LLMs, monolingual corpora, and a scoring function to create high-quality, diversified IFT datasets in multiple languages. Experiments demonstrate that LLMs finetuned using these IFT datasets show notable improvements in both generative and discriminative tasks, indicating enhanced language comprehension by LLMs in non-English contexts. Specifically, on the multilingual summarization task, LLMs using our IFT dataset achieved 17.57% and 15.23% improvements over LLMs fine-tuned with translation-based and template-based datasets, respectively.

cs.CL↗

arXiv2024

On the Expressive Power of Spectral Invariant Graph Neural Networks

Incorporating spectral information to enhance Graph Neural Networks (GNNs) has shown promising results but raises a fundamental challenge due to the inherent ambiguity of eigenvectors. Various architectures have been proposed to address this ambiguity, referred to as spectral invariant architectures. Notable examples include GNNs and Graph Transformers that use spectral distances, spectral projection matrices, or other invariant spectral features. However, the potential expressive power of these spectral invariant architectures remains largely unclear. The goal of this work is to gain a deep theoretical understanding of the expressive power obtainable when using spectral features. We first introduce a unified message-passing framework for designing spectral invariant GNNs, called Eigenspace Projection GNN (EPNN). A comprehensive analysis shows that EPNN essentially unifies all prior spectral invariant architectures, in that they are either strictly less expressive or equivalent to EPNN. A fine-grained expressiveness hierarchy among different architectures is also established. On the other hand, we prove that EPNN itself is bounded by a recently proposed class of Subgraph GNNs, implying that all these spectral invariant architectures are strictly less expressive than 3-WL. Finally, we discuss whether using spectral features can gain additional expressiveness when combined with more expressive GNNs.

cs.LG↗

arXiv2024

Disorder-broadened phase boundary with enhanced amorphous superconductivity in pressurized In2Te5

As an empirical tool in materials science and engineering, the iconic phase diagram owes its robustness and practicality to the topological characteristics rooted in the celebrated Gibbs phase law (F = C - P + 2). When crossing the phase diagram boundary, the structure transition occurs abruptly, bringing about an instantaneous change in physical properties and limited controllability on the boundaries (F = 1). Here, we expand the sharp phase boundary to an amorphous transition region (F = 2) by partially disrupting the long-range translational symmetry, leading to a sequential crystalline-amorphous-crystalline (CAC) transition in a pressurized In2Te5 single crystal. Through detailed in-situ synchrotron diffraction, we elucidate that the phase transition stems from the rotation of immobile blocks [In2Te2]2+, linked by hinge-like [Te3]2- trimers. Remarkably, within the amorphous region, the amorphous phase demonstrates a notable 25 % increase of the superconducting transition temperature (Tc), while the carrier concentration remains relatively constant. Furthermore, we propose a theoretical framework revealing that the unconventional boost in amorphous superconductivity might be attributed to an intensified electron correlation, triggered by a disorder-augmented multifractal behavior. These findings underscore the potential of disorder and prompt further exploration of unforeseen phenomena on the phase boundaries.

cond-mat.supr-con↗

arXiv2024

Descriptive Kernel Convolution Network with Improved Random Walk Kernel

Graph kernels used to be the dominant approach to feature engineering for structured data, which are superseded by modern GNNs as the former lacks learnability. Recently, a suite of Kernel Convolution Networks (KCNs) successfully revitalized graph kernels by introducing learnability, which convolves input with learnable hidden graphs using a certain graph kernel. The random walk kernel (RWK) has been used as the default kernel in many KCNs, gaining increasing attention. In this paper, we first revisit the RWK and its current usage in KCNs, revealing several shortcomings of the existing designs, and propose an improved graph kernel RWK+, by introducing color-matching random walks and deriving its efficient computation. We then propose RWK+CN, a KCN that uses RWK+ as the core kernel to learn descriptive graph features with an unsupervised objective, which can not be achieved by GNNs. Further, by unrolling RWK+, we discover its connection with a regular GCN layer, and propose a novel GNN layer RWK+Conv. In the first part of experiments, we demonstrate the descriptive learning ability of RWK+CN with the improved random walk kernel RWK+ on unsupervised pattern mining tasks; in the second part, we show the effectiveness of RWK+ for a variety of KCN architectures and supervised graph learning tasks, and demonstrate the expressiveness of RWK+Conv layer, especially on the graph-level tasks. RWK+ and RWK+Conv adapt to various real-world applications, including web applications such as bot detection in a web-scale Twitter social network, and community classification in Reddit social interaction networks.

cs.LG↗

arXiv2023

Pressure-induced Superconductivity in Tellurium Single Crystal

Tellurium (Te) is one of the p-orbital chalcogens, which shows attractive physical properties at ambient pressure. Here, we systematically investigate both structural and electronic evolution of Te single crystal under high pressure up to 40 GPa. The pressure dependence of the experimental Raman spectrum reveals the occurrence of multiple phase transitions, which is consistent with highpressure synchrotron X-ray diffraction (XRD) measurements. The appearance of superconductivity in high-pressure phase of Te is accompanied by structural phase transitions. The high-pressure phases of Te reveal a nonmonotonic evolution of superconducting temperature Tc with notably different upper critical fields. The theoretical calculations demonstrate that the pressure dependence of the density of states(DOS) agrees well with the variation of Tc. Our results provide a systematic phase diagram for the pressure-induced superconductivity of Te.

cond-mat.supr-con↗

arXiv2023

ADAMM: Anomaly Detection of Attributed Multi-graphs with Metadata: A Unified Neural Network Approach

Given a complex graph database of node- and edge-attributed multi-graphs as well as associated metadata for each graph, how can we spot the anomalous instances? Many real-world problems can be cast as graph inference tasks where the graph representation could capture complex relational phenomena (e.g., transactions among financial accounts in a journal entry), along with metadata reflecting tabular features (e.g. approver, effective date, etc.). While numerous anomaly detectors based on Graph Neural Networks (GNNs) have been proposed, none are capable of directly handling directed graphs with multi-edges and self-loops. Furthermore, the simultaneous handling of relational and tabular features remains an unexplored area. In this work we propose ADAMM, a novel graph neural network model that handles directed multi-graphs, providing a unified end-to-end architecture that fuses metadata and graph-level representation learning through an unsupervised anomaly detection objective. Experiments on datasets from two different domains, namely, general-ledger journal entries from different firms (accounting) as well as human GPS trajectories from thousands of individuals (urban mobility) validate ADAMM's generality and detection effectiveness of expert-guided and ground-truth anomalies. Notably, ADAMM outperforms existing baselines that handle the two data modalities (graph and metadata) separately with post hoc synthesis efforts.

cs.LG↗

arXiv2023

Realistic Volume Rendering with Environment-Synced Illumination in Mixed Reality

Interactive volume visualization using a mixed reality (MR) system helps provide users with an intuitive spatial perception of volumetric data. Due to sophisticated requirements of user interaction and vision when using MR head-mounted display (HMD) devices, the conflict between the realisticness and efficiency of direct volume rendering (DVR) is yet to be resolved. In this paper, a new MR visualization framework that supports interactive realistic DVR is proposed. An efficient illumination estimation method is used to identify the high dynamic range (HDR) environment illumination captured using a panorama camera. To improve the visual quality of Monte Carlo-based DVR, a new spatio-temporal denoising algorithm is designed. Based on a reprojection strategy, it makes full use of temporal coherence between adjacent frames and spatial coherence between the two screens of an HMD to optimize MR rendering quality. Several MR development modules are also developed for related devices to efficiently and stably display the DVR results in an MR HMD. Experimental results demonstrate that our framework can better support immersive and intuitive user perception during MR viewing than existing MR solutions.

cs.GR↗