arXiv subjects

Zhiyuan Li

Publications and source records attributed to Zhiyuan Li.

At least 145 records · Page 8Linked to original sources

arXiv2025

Kimi Linear: An Expressive, Efficient Attention Architecture

We introduce Kimi Linear, a hybrid linear attention architecture that, for the first time, outperforms full attention under fair comparisons across various scenarios -- including short-context, long-context, and reinforcement learning (RL) scaling regimes. At its core lies Kimi Delta Attention (KDA), an expressive linear attention module that extends Gated DeltaNet with a finer-grained gating mechanism, enabling more effective use of limited finite-state RNN memory. Our bespoke chunkwise algorithm achieves high hardware efficiency through a specialized variant of the Diagonal-Plus-Low-Rank (DPLR) transition matrices, which substantially reduces computation compared to the general DPLR formulation while remaining more consistent with the classical delta rule. We pretrain a Kimi Linear model with 3B activated parameters and 48B total parameters, based on a layerwise hybrid of KDA and Multi-Head Latent Attention (MLA). Our experiments show that with an identical training recipe, Kimi Linear outperforms full MLA with a sizeable margin across all evaluated tasks, while reducing KV cache usage by up to 75% and achieving up to 6 times decoding throughput for a 1M context. These results demonstrate that Kimi Linear can be a drop-in replacement for full attention architectures with superior performance and efficiency, including tasks with longer input and output lengths. To support further research, we open-source the KDA kernel and vLLM implementations, and release the pre-trained and instruction-tuned model checkpoints.

cs.CL↗

arXiv2025

Radio signatures of AGN-wind-driven shocks in elliptical galaxies: From simulations to observations

We investigate the synchrotron emission signatures of shocks driven by active galactic nucleus (AGN) wind in elliptical galaxies based on our two-dimensional axisymmetric hydrodynamic MACER numerical simulations. Using these simulation data, we calculate the synchrotron radiation produced by nonthermal electrons accelerated at shocks, adopting reasonable assumptions for the magnetic field and relativistic electron distribution (derived from diffusive shock acceleration theory), and predict the resulting observational signatures. In our fiducial model, shocks driven by AGN winds produce synchrotron emission with luminosities of approximately $10^{29}\,\mathrm{erg\,s^{-1}\,Hz^{-1}}$ in the radio band (0.5-5 GHz), with spectral indices of $α\approx -0.4$ to $-0.6$ during the strongest shock phases, gradually steepening to about $-0.8$ to $-1.4$ as the electron population ages. Spatially, the emission is initially concentrated in regions of strong shocks, later expanding into more extended, diffuse structures. We also apply our model to the dwarf elliptical galaxy Messier 32 (M32), and find remarkable consistency between our simulated emission and the observed nuclear radio source, suggesting that this radio component likely originates from hot-wind-driven shocks. Our results indicate that AGN winds not only influence galaxy gas dynamics through mechanical energy input but also yield direct observational evidence via nonthermal radiation. With the advent of next-generation radio facilities such as the FAST Core Array, SKA, and ngVLA, these emission signatures serve as important probes for detecting and characterizing AGN feedback.

astro-ph.GA↗

arXiv2025

Projective Equivalence of Smooth Hypersurfaces via Cyclic Covers

In this paper, we prove that for any smooth hypersurface $Y$ of degree $d$ in $\mathbb{P}^{n+1}_k$, the cyclic $d$-fold cover $\widetilde{Y} \to \mathbb{P}^{n+1}_k$ branched along $Y$ completely characterizes $Y$ up to projective equivalence. This solves a question asked by Huybrechts in [Huy23, §1.5.6].

math.AG↗

arXiv2025

Inverse source problem with a posteriori interior measurements for space-time fractional diffusion equations

This paper investigates an inverse source problem for space-time fractional diffusion equations from a posteriori interior measurements. The uniqueness result is established by the memory effect of fractional derivatives and the unique continuation property. For the numerical reconstruction, the inverse problem is reformulated as an optimization problem with the Tikhonov regularization. We use the Levenberg-Marquardt method to identity the unknown source from noisy measurements. Finally, we give some numerical examples to illustrate the efficiency and accuracy of the proposed algorithm.

math.NA↗

arXiv2025

ReXMoE: Reusing Experts with Minimal Overhead in Mixture-of-Experts

Mixture-of-Experts (MoE) architectures have emerged as a promising approach to scale Large Language Models (LLMs). MoE boosts the efficiency by activating a subset of experts per token. Recent works show that fine-grained experts substantially enriches the combinatorial flexibility of active experts and enhances model expressiveness. However, such a design is fundamentally limited by the layer-local routing mechanism: each layer is restricted to its own expert pool. This requires a careful trade-off between expert dimensionality and routing diversity given fixed parameter budgets. We describe ReXMoE, a novel MoE architecture that improves routing beyond the existing layer-local approaches by allowing routers to reuse experts across adjacent layers. ReXMoE decouples expert dimensionality from per-layer budgets, enabling richer expert combinations without sacrificing individual expert capacity or inflating overall parameters. To this end, we propose a new progressive scaling routing (PSR) strategy to gradually increase the candidate expert pool during training. As a result, ReXMoE improves both language modeling and downstream task performance. Extensive experiments on models ranging from 0.5B to 7B parameters across different architectures demonstrate that ReXMoE consistently improves performance under fixed architectural dimensions, confirming ReXMoE as new design paradigm for parameter-efficient and scalable MoE-based LLMs.

cs.CL↗

arXiv2025

Sharp decay estimates and numerical analysis for weakly coupled systems of two subdiffusion equations

This paper investigates the initial-boundary value problem for weakly coupled systems of time-fractional subdiffusion equations with spatially and temporally varying coupling coefficients. By combining the energy method with the coercivity of fractional derivatives, we convert the original partial differential equations into a coupled ordinary differential system. Through Laplace transform and maximum principle arguments, we reveal a dichotomy in decay behavior: When the highest fractional order is less than one, solutions exhibit sublinear decay, whereas systems with the highest order equal to one demonstrate a distinct superlinear decay pattern. This phenomenon fundamentally distinguishes coupled systems from single fractional diffusion equations, where such accelerated superlinear decay never occurs. Numerical experiments employing finite difference methods and implicit discretization schemes validate the theoretical findings.

math.AP↗

arXiv2025

Optimistic Multi-Agent Policy Gradient

*Relative overgeneralization* (RO) occurs in cooperative multi-agent learning tasks when agents converge towards a suboptimal joint policy due to overfitting to suboptimal behavior of other agents. No methods have been proposed for addressing RO in multi-agent policy gradient (MAPG) methods although these methods produce state-of-the-art results. To address this gap, we propose a general, yet simple, framework to enable optimistic updates in MAPG methods that alleviate the RO problem. Our approach involves clipping the advantage to eliminate negative values, thereby facilitating optimistic updates in MAPG. The optimism prevents individual agents from quickly converging to a local optimum. Additionally, we provide a formal analysis to show that the proposed method retains optimality at a fixed point. In extensive evaluations on a diverse set of tasks including the *Multi-agent MuJoCo* and *Overcooked* benchmarks, our method outperforms strong baselines on 13 out of 19 tested tasks and matches the performance on the rest.

cs.LG↗

arXiv2025

Generate Logical Equivalence Questions

Academic dishonesty is met with zero tolerance in higher education, yet plagiarism has become increasingly prevalent in the era of online teaching and learning. Automatic Question Generation (AQG) presents a potential solution to mitigate copying by creating unique questions for each student. Additionally, AQG can provide a vast array of practice questions. Our AQG focuses on generating logical equivalence questions for Discrete Mathematics, a foundational course for first-year computer science students. A literature review reveals that existing AQGs for this type of question generate all propositions that meet user-defined constraints, resulting in inefficiencies and a lack of uniform question difficulty. To address this, we propose a new approach that defines logical equivalence questions using a formal language, translates this language into two sets of generation rules, and develops a linear-time algorithm for question generation. We evaluated our AQG through two experiments. The first involved a group of students completing questions generated by our system. Statistical analysis shows that the accuracy of these questions is comparable to that of textbook questions. The second experiment assessed the number of steps required to solve our generated questions, textbook questions, and those generated by multiple large language models. The results indicated that the difficulty of our questions was similar to that of textbook questions, confirming the quality of our AQG.

cs.CL↗

arXiv2025

Advancing Marine Research: UWSAM Framework and UIIS10K Dataset for Precise Underwater Instance Segmentation

With recent breakthroughs in large-scale modeling, the Segment Anything Model (SAM) has demonstrated significant potential in a variety of visual applications. However, due to the lack of underwater domain expertise, SAM and its variants face performance limitations in end-to-end underwater instance segmentation tasks, while their higher computational requirements further hinder their application in underwater scenarios. To address this challenge, we propose a large-scale underwater instance segmentation dataset, UIIS10K, which includes 10,048 images with pixel-level annotations for 10 categories. Then, we introduce UWSAM, an efficient model designed for automatic and accurate segmentation of underwater instances. UWSAM efficiently distills knowledge from the SAM ViT-Huge image encoder into the smaller ViT-Small image encoder via the Mask GAT-based Underwater Knowledge Distillation (MG-UKD) method for effective visual representation learning. Furthermore, we design an End-to-end Underwater Prompt Generator (EUPG) for UWSAM, which automatically generates underwater prompts instead of explicitly providing foreground points or boxes as prompts, thus enabling the network to locate underwater instances accurately for efficient segmentation. Comprehensive experimental results show that our model is effective, achieving significant performance improvements over state-of-the-art methods on multiple underwater instance datasets. Datasets and codes are available at https://github.com/LiamLian0727/UIIS10K.

cs.CV↗

arXiv2025

Episodic Feedback in Triple AGN Candidate SDSS J0849+1114 Revealed by Extended ionized gas

Galaxy mergers funnel gas toward the nuclei, igniting starbursts and active galactic nuclei (AGNs). The AGN feedback can reshape the host galaxy and regulate both star formation and super-massive black-hole (SMBH) accretion. Using VLT/MUSE integral-field spectroscopy, we conduct a spatially resolved study of the triple-AGN candidate SDSS J0849+1114. Extended ionized gas structures ($>10$ kpc from nucleus A) primarily associated with tidal tails are detected. Meanwhile, two distinct ionized gas outflows are revealed. One extends over $>5$ kpc around nuclei A with a kinetic power of $\dot{E}_{\rm out,A} = 3.0\times10^{42}\rm\, erg\, s^{-1}$, which might be driven by the radio jet. The other outflow extends $\sim 5.9$ kpc around nucleus C, with a kinetic power of $\dot{E}_{\rm out,C} = 2.0\times10^{40}\rm\, erg\, s^{-1}$. High [O III]/H$α$ and [N II]/H$α$ ratios in the tidal gas require that nucleus A radiated at a high accretion rate with $L_{\rm A,bol} \sim 0.1$--$0.5\,L_{\rm Edd,A}$ at least $\sim3$--$\times10^{4}\rm\,yr$ ago, $20$--$100$ times brighter than today. Combined with multi-wavelength constraints, we find evidence for episodic AGN feedback that expelled circumnuclear gas and rapidly quenched accretion. This triple AGN candidate demonstrates how AGN feedback can self-regulate black hole growth and impact hosts during mergers.

astro-ph.GA↗

arXiv2025

Pitfalls of defacing whole-head MRI: re-identification risk with diffusion models and compromised research potential

Defacing is often applied to head magnetic resonance image (MRI) datasets prior to public release to address privacy concerns. The alteration of facial and nearby voxels has provoked discussions about the true capability of these techniques to ensure privacy as well as their impact on downstream tasks. With advancements in deep generative models, the extent to which defacing can protect privacy is uncertain. Additionally, while the altered voxels are known to contain valuable anatomical information, their potential to support research beyond the anatomical regions directly affected by defacing remains uncertain. To evaluate these considerations, we develop a refacing pipeline that recovers faces in defaced head MRIs using cascaded diffusion probabilistic models (DPMs). The DPMs are trained on images from 180 subjects and tested on images from 484 unseen subjects, 469 of whom are from a different dataset. To assess whether the altered voxels in defacing contain universally useful information, we also predict computed tomography (CT)-derived skeletal muscle radiodensity from facial voxels in both defaced and original MRIs. The results show that DPMs can generate high-fidelity faces that resemble the original faces from defaced images, with surface distances to the original faces significantly smaller than those of a population average face (p < 0.05). This performance also generalizes well to previously unseen datasets. For skeletal muscle radiodensity predictions, using defaced images results in significantly weaker Spearman's rank correlation coefficients compared to using original images (p < 10-4). For shin muscle, the correlation is statistically significant (p < 0.05) when using original images but not statistically significant (p > 0.05) when any defacing method is applied, suggesting that defacing might not only fail to protect privacy but also eliminate valuable information.

eess.IV↗

arXiv2025

Brain age identification from diffusion MRI synergistically predicts neurodegenerative disease

Estimated brain age from magnetic resonance image (MRI) and its deviation from chronological age can provide early insights into potential neurodegenerative diseases, supporting early detection and implementation of prevention strategies. Diffusion MRI (dMRI) presents an opportunity to build an earlier biomarker for neurodegenerative disease prediction because it captures subtle microstructural changes that precede more perceptible macrostructural changes. However, the coexistence of macro- and micro-structural information in dMRI raises the question of whether current dMRI-based brain age estimation models are leveraging the intended microstructural information or if they inadvertently rely on the macrostructural information. To develop a microstructure-specific brain age, we propose a method for brain age identification from dMRI that mitigates the model's use of macrostructural information by non-rigidly registering all images to a standard template. Imaging data from 13,398 participants across 12 datasets were used for the training and evaluation. We compare our brain age models, trained with and without macrostructural information mitigated, with an architecturally similar T1-weighted (T1w) MRI-based brain age model and two recent, popular, openly available T1w MRI-based brain age models that primarily use macrostructural information. We observe difference between our dMRI-based brain age and T1w MRI-based brain age across stages of neurodegeneration, with dMRI-based brain age being older than T1w MRI-based brain age in participants transitioning from cognitively normal (CN) to mild cognitive impairment (MCI), but younger in participants already diagnosed with Alzheimer's disease (AD). Furthermore, dMRI-based brain age may offer advantages over T1w MRI-based brain age in predicting the transition from CN to MCI up to five years before diagnosis.

cs.CV↗

arXiv2025

DP-FedLoRA: Privacy-Enhanced Federated Fine-Tuning for On-Device Large Language Models

As on-device large language model (LLM) systems become increasingly prevalent, federated fine-tuning enables advanced language understanding and generation directly on edge devices; however, it also involves processing sensitive, user-specific data, raising significant privacy concerns within the federated learning framework. To address these challenges, we propose DP-FedLoRA, a privacy-enhanced federated fine-tuning framework that integrates LoRA-based adaptation with differential privacy in a communication-efficient setting. Each client locally clips and perturbs its LoRA matrices using Gaussian noise to satisfy ($ε$, $δ$)-differential privacy. We further provide a theoretical analysis demonstrating the unbiased nature of the updates and deriving bounds on the variance introduced by noise, offering practical guidance for privacy-budget calibration. Experimental results across mainstream benchmarks show that DP-FedLoRA delivers competitive performance while offering strong privacy guarantees, paving the way for scalable and privacy-preserving LLM deployment in on-device environments.

cs.CR↗

arXiv2025

Discovery of giant bubbles in the hot gaseous halo of the massive disk galaxy NGC 6286

Based on archival Chandra X-ray observation, optical integral-field spectroscopic data and radio interferometric data, we report the discovery of a pair of giant bubbles (with a projected radius ~ 5 kpc) of ionized gas emerging from a highly inclined starburst galaxy NGC 6286, which is undergoing strong tidal interactions with its close neighbor NGC 6285. The bubbles are manifested by extraplanar soft X-ray emission with an X-shaped morphology, which is tightly co-spatial with $\rm Hα$ line emission and partially coincident with radio continuum. Low surface brightness diffuse X-ray emission can be traced out to ~ 90 kpc from the galactic center, revealing the presence of an extended hot gaseous halo. X-ray spectral analysis of the bubbles yields a gas temperature of $0.70^{+0.16}_{-0.18}$ keV, a relatively high value among known galactic-scale bubbles in the local universe. An average energy injection rate of ~ $10^{43}$ erg $\rm s^{-1}$ is required to inflate the bubbles within an estimated dynamical age of ~ 6.4 Myr. The multi-wavelength properties of the bubbles can be understood with the conventional superwind scenario, in which disk/halo gas is swept up into an expanding, cooling shell by a hot tenuous wind. The current starburst in NGC 6286 is energetically sufficient to launch the superwind, although we cannot rule out the possibility of a more violent AGN in the recent past as the driving source. Future high-resolution spectroscopic observations will help to shed light on the origin of the superwind and its role as an important galactic feedback process.

astro-ph.GA↗

arXiv2025

Hölder Stable Recovery of the Source in Space-Time Fractional Wave Equations

We study the recovery of a spatially dependent source in a one-dimensional space-time fractional wave equation using boundary measurement data collected at a single endpoint. The main challenge arises from the fact that the eigenfunctions of the Dirichlet eigenvalue problem do not form an orthogonal system, due to the presence of a fractional derivative in space. To address this difficulty, we introduce a bi-orthogonal basis for the Mittag-Leffler functions and use it to establish uniqueness and Hölder-type stability results, provided the measurement time is sufficiently large. A Tikhonov regularization method is then employed to numerically solve the inverse source problem. Several numerical examples are presented to demonstrate the accuracy and efficiency of the proposed method and to validate our theoretical findings.

math.AP↗

arXiv2025

A Survey: Towards Privacy and Security in Mobile Large Language Models

Mobile Large Language Models (LLMs) are revolutionizing diverse fields such as healthcare, finance, and education with their ability to perform advanced natural language processing tasks on-the-go. However, the deployment of these models in mobile and edge environments introduces significant challenges related to privacy and security due to their resource-intensive nature and the sensitivity of the data they process. This survey provides a comprehensive overview of privacy and security issues associated with mobile LLMs, systematically categorizing existing solutions such as differential privacy, federated learning, and prompt encryption. Furthermore, we analyze vulnerabilities unique to mobile LLMs, including adversarial attacks, membership inference, and side-channel attacks, offering an in-depth comparison of their effectiveness and limitations. Despite recent advancements, mobile LLMs face unique hurdles in achieving robust security while maintaining efficiency in resource-constrained environments. To bridge this gap, we propose potential applications, discuss open challenges, and suggest future research directions, paving the way for the development of trustworthy, privacy-compliant, and scalable mobile LLM systems.

cs.CR↗

arXiv2025

A Theory of Learning with Autoregressive Chain of Thought

For a given base class of sequence-to-next-token generators, we consider learning prompt-to-answer mappings obtained by iterating a fixed, time-invariant generator for multiple steps, thus generating a chain-of-thought, and then taking the final token as the answer. We formalize the learning problems both when the chain-of-thought is observed and when training only on prompt-answer pairs, with the chain-of-thought latent. We analyze the sample and computational complexity both in terms of general properties of the base class (e.g. its VC dimension) and for specific base classes such as linear thresholds. We present a simple base class that allows for universal representability and computationally tractable chain-of-thought learning. Central to our development is that time invariance allows for sample complexity that is independent of the length of the chain-of-thought. Attention arises naturally in our construction.

stat.ML↗

arXiv2025

Characterizing Visual Intents for People with Low Vision through Eye Tracking

Accessing visual information is crucial yet challenging for people with low vision due to visual conditions like low visual acuity and limited visual fields. However, unlike blind people, low vision people have and prefer using their functional vision in daily tasks. Gaze patterns thus become an important indicator to uncover their visual challenges and intents, inspiring more adaptive visual support. We seek to deeply understand low vision users' gaze behaviors in different image-viewing tasks, characterizing typical visual intents and the unique gaze patterns exhibited by people with different low vision conditions. We conducted a retrospective think-aloud study using eye tracking with 20 low vision participants and 20 sighted controls. Participants completed various image-viewing tasks and watched the playback of their gaze trajectories to reflect on their visual experiences. Based on the study, we derived a visual intent taxonomy with five visual intents characterized by participants' gaze behaviors. We demonstrated the difference between low vision and sighted participants' gaze behaviors and how visual ability affected low vision participants' gaze patterns across visual intents. Our findings underscore the importance of combining visual ability information, visual context, and eye tracking data in visual intent recognition, setting up a foundation for intent-aware assistive technologies for low vision people.

cs.HC↗