arXiv subjects

Yihang Zhang

Publications and source records attributed to Yihang Zhang.

At least 19 recordsLinked to original sources

arXiv2026

Escaping Low-Dimensional Overlap: Multi-Task Model Merging via High-Dimensional Sparse Disentanglement

Model merging provides an efficient way to construct multi-task generalist models without additional training, but its performance often degrades under severe task interference. Task interference in model merging primarily stems from \textit{superposition}, where task-specific features become entangled within the parameter space. This entanglement renders conventional decomposition methods insufficient for effectively isolating useful task directions from interfering components. In this paper, we propose a sparse-representation-based merging framework that uses Sparse Autoencoders (SAEs) to project task vectors into a high-dimensional sparse feature space, enabling feature-level disentanglement before fusion. To reduce computational overhead, we further introduce a lightweight Group-Ranked Zeroth-Order Optimizer (GR-ZOO) to identify task-critical layers for selective merging. Experiments on both Qwen2.5-1.5B and Qwen2.5-7B demonstrate that our method consistently outperforms representative baselines, including Task Arithmetic, TIES-Merge, DARE, Fisher-Merge,and several recent training-free merging methods, across mathematical reasoning, code generation, instruction following, and general knowledge tasks. In a highly conflicting four-task setting on Qwen2.5-1.5B, our method further achieves a 2.78\% improvement over the strongest baseline.

cs.LG↗

arXiv2026

Evaluating and Pricing Advertisements in AI-Generated Responses

As search increasingly shifts toward LLM-driven answer engines, advertising is becoming embedded within the generated response itself and should therefore be evaluated for both user utility and commercial value. The key challenge is click-through intent: behavioural logs are unavailable, human annotation resists calibration, and frontier LLM judges conflate intent with linguistic fluency. These gaps compound, as principled pricing presupposes a continuous intent signal, while generating such a signal presupposes supervision that is currently unavailable. We construct the missing supervision through a psychologically grounded agent simulation framework, and distil it into a parameter-efficient evaluator that predicts click-through intent, together with the three companion dimensions of ad quality, as smooth, differentiable estimates. Validated through sign-certain behavioural perturbations, the evaluator surpasses frontier zero-shot judges on relevance sensitivity (79% versus 60-67%), tracks graded content degradation, generalises without error to 103 fictional products, and agrees with human preference in 86% of pairwise judgements across five annotators, with agreement rising in the evaluator's confidence. Upon its estimates we build the pricing layer directly, deriving the unique payment rule under which truthful bidding is optimal, demonstrating it on a best-of-k allocation, and extending the mechanism to non-monotone allocations. The same differentiable signal stands ready as a training objective for ad generation.

cs.AI↗

arXiv2026

PILA: Plug-and-Play Insertion for LLM-native Advertising

How to monetize large language models (LLMs) by naturally integrating sponsored content into their responses, known as LLM-native advertising, has recently emerged as a critical problem. However, existing solutions entangle advertising with content generation inside a single model, which is incompatible with modern API-only or workflow-based LLM applications and inevitably compromises the original response quality. To address this, we propose PILA, which reformulates ad insertion as a conditional response rewriting problem and decouples it from the upstream service as a lightweight sidecar module. PILA is model-agnostic and can be seamlessly integrated with existing LLM services without modifying the base model or its workflow. It also exposes a controllable trade-off between user-side naturalness and ad-side exposure, offering a practical interface for downstream pricing and deployment. Experiments across diverse upstream models show that \pila consistently improves ad effectiveness while preserving response quality, highlighting its promise as a practical solution for LLM-native advertising.

cs.CL↗

arXiv2026

A Jet from a Nearly Dormant Black Hole

Most galaxies host supermassive black holes (SMBHs) that remain weakly accreting or dormant for much of their lifetimes. At the lowest accretion rates, these systems may represent the transition between active nuclei and dormant black holes, but whether they can still launch collimated jets remains unclear. The nuclei in our Galaxy (\sgra) and M31 are key examples of this regime, although no clear jet structure has yet been detected in either source. Here we report multi-frequency very long baseline interferometric observations of \Msixty\ (NGC~4649), a nearby elliptical galaxy hosting a nearly dormant SMBH with an Eddington ratio of $\sim10^{-8}$. We detect a compact two-sided jet with an unusually steep synchrotron spectrum, demonstrating that collimated outflows can persist even under nearly dormant accretion conditions. The apparent radio core exhibits an unprecedentedly steep frequency-dependent position shift toward the SMBH, locating the central engine only $\sim57\,\mu$as, corresponding to a projected distance of $\sim10$ Schwarzschild radii, upstream of the 8.37-GHz core. The observed jet morphology and steep core-shift behaviour are reproduced by general relativistic magnetohydrodynamic and radiative-transfer simulations, indicating a magnetically dominated, non-equipartition jet-launching region that departs from the standard conical equipartition picture. These results provide direct observational evidence that jet production can survive near the dormant SMBHs and establish \Msixty\ as a unique laboratory for probing jet formation on event-horizon scales in the lowest-accretion SMBH regime.

astro-ph.HE↗

arXiv2026

NaiAD: Initiate Data-Driven Research for LLM Advertising

Reconciling platform revenue with user experience in LLM advertising motivates a data-centric foundation. We introduce NaiAD, the first comprehensive dataset for LLM-native advertising comprising 58,999 carefully constructed ad-embedded responses paired with user queries. NaiAD is organized around theoretically grounded evaluation metrics that separately and comprehensively capture user and commercial utility. To mitigate the dimensional collinearity of aligned LLMs, we propose a decoupled generation pipeline that produces structurally diverse samples, ranging from responses that explicitly disentangle stakeholder utilities to responses that are uniformly strong or weak across dimensions. We further provide score labels calibrated by a Variance-Calibrated Prediction-Powered Inference (VC-PPI) framework, aligning automated scoring with human annotations. Mechanistic analyses reveal that successful ad integration relies on reasoning paths that cluster into four distinct semantic strategies. Models leveraging NaiAD internalize these strategies to simultaneously improve user and commercial utility, while enabling independent control over these distinct objectives via in-context learning. Together, these results position NaiAD as a foundational infrastructure for developing future LLM-native ad systems.

cs.LG↗

arXiv2026

LLM Advertisement based on Neuron Auctions

As Large Language Models (LLMs) transition into conversational agents, generative advertising emerges as a crucial monetization strategy. However, embedding advertisements within unstructured LLM outputs introduces a critical trilemma: balancing advertiser payoffs, platform revenue, and user experience. Existing methods, such as prompt injection or rigid position slots, disrupt semantic coherence and lack a parametric framework for independent control, rendering rigorous mechanism design intractable. To bridge this gap, we introduce Neuron Auctions, a novel paradigm that shifts the auction object from the surface text space to the LLM's internal representations. Leveraging mechanistic interpretability, we identify brand-specific feed-forward network (FFN) neurons and demonstrate that competing brands activate within approximately orthogonal subspaces. This near-perfect independence allows us to define continuous, disentangled intervention budgets (specifically, neuron counts and amplification factors) as auctionable commodities. Building on this computational carrier, we design a continuous menu-based auction mechanism that naturally guarantees strategy-proofness and optimizes revenue for the platform. By explicitly incorporating a user utility penalty into the platform's optimization objective, our framework dynamically prices out overly aggressive interventions. Extensive experiments demonstrate that Neuron Auctions effectively preserve natural discourse quality while achieving an optimal alignment between commercial incentives and user satisfaction.

cs.LG↗

arXiv2025

Reverse Thinking Enhances Missing Information Detection in Large Language Models

Large Language Models (LLMs) have demonstrated remarkable capabilities in various reasoning tasks, yet they often struggle with problems involving missing information, exhibiting issues such as incomplete responses, factual errors, and hallucinations. While forward reasoning approaches like Chain-of-Thought (CoT) and Tree-of-Thought (ToT) have shown success in structured problem-solving, they frequently fail to systematically identify and recover omitted information. In this paper, we explore the potential of reverse thinking methodologies to enhance LLMs' performance on missing information detection tasks. Drawing inspiration from recent work on backward reasoning, we propose a novel framework that guides LLMs through reverse thinking to identify necessary conditions and pinpoint missing elements. Our approach transforms the challenging task of missing information identification into a more manageable backward reasoning problem, significantly improving model accuracy. Experimental results demonstrate that our reverse thinking approach achieves substantial performance gains compared to traditional forward reasoning methods, providing a promising direction for enhancing LLMs' logical completeness and reasoning robustness.

cs.AI↗

arXiv2025

DFAMS: Dynamic-flow guided Federated Alignment based Multi-prototype Search

Federated Retrieval (FR) routes queries across multiple external knowledge sources, to mitigate hallucinations of LLMs, when necessary external knowledge is distributed. However, existing methods struggle to retrieve high-quality and relevant documents for ambiguous queries, especially in cross-domain scenarios, which significantly limits their effectiveness in supporting downstream generation tasks. Inspired by Dynamic Information Flow (DIF), we propose DFAMS, a novel framework that leverages DIF to identify latent query intents and construct semantically aligned knowledge partitions for accurate retrieval across heterogeneous sources. Specifically, DFAMS probes the DIF in LLMs by leveraging gradient signals from a few annotated queries and employing Shapley value-based attribution to trace neuron activation paths associated with intent recognition and subdomain boundary detection. Then, DFAMS leverages DIF to train an alignment module via multi-prototype contrastive learning, enabling fine-grained intra-source modeling and inter-source semantic alignment across knowledge bases. Experimental results across five benchmarks show that DFAMS outperforms advanced FR methods by up to 14.37\% in knowledge classification accuracy, 5.38\% in retrieval recall, and 6.45\% in downstream QA accuracy, demonstrating its effectiveness in complex FR scenarios. Our code are anonymous available at https://anonymous.4open.science/r/DFAMS/

cs.LG↗

arXiv2025

Multi-timescale Stochastic Programming with Applications in Power Systems

This paper introduces a multi-timescale stochastic programming framework designed to address decision-making challenges in power systems, particularly those with high renewable energy penetration. The framework models interactions across different timescales using aggregated state variables to coordinate decisions. In addition to Multi-timescale uncertainty modeled via multihorizon trees, we also introduce a "synchronized state approximation," which periodically aligns states across timescales to maintain consistency and tractability. Using this approximation, we propose two instantiation methods: a scenario-based approach and a value function-based approach specialized for this setup. Our framework is very generic, and covers a wide-spectrum of applications.

math.OC↗

arXiv2025

Production of Iodine Isotopes via Ultra-intense Laser Driven Photonuclear Reactions

The investigation and production of proton-rich iodine isotopes predominantly rely on conventional accelerator-based methods, typically requiring prolonged irradiation periods to measure or achieve quantifiable yields for isotopic isolation. Bremsstrahlung radiation sources generated by high-power laser-plasma-accelerated electron beams with ultrahigh charge (tens of nanocoulombs) bombarding high-Z targets demonstrate extraordinary photon flux characteristics. An electron beam with a total charge of approximately 47.7 nC (E$_e$ $\gt$ 10.4 MeV) was generated in our experiment by focusing a ultra-intense laser pulse onto a deuterium gas jet. Laser-driven bremsstrahlung was employed to induce $^{127}I$$(\gamma,xn)$ ($x$ = 1,3,4,6-8), and the product yields and the corresponding flux-weighted average cross sections are reported. Our results demonstrate production of medical isotopes, with average yields of $^{124}$I and $^{123}$I at approximately $9.83\pm0.45\times10^{5}$/shot and $2.81\pm0.11\times10^{5}$/shot, respectively. This method, utilizing high-power lasers to generate bremsstrahlung radiation, shows significant potential for medical applications and opens new avenues for studying photonuclear processes in astrophysical contexts.

nucl-ex↗

arXiv2025

Beyond First-Order: Training LLMs with Stochastic Conjugate Subgradients and AdamW

Stochastic gradient-based descent (SGD), have long been central to training large language models (LLMs). However, their effectiveness is increasingly being questioned, particularly in large-scale applications where empirical evidence suggests potential performance limitations. In response, this paper proposes a stochastic conjugate subgradient method together with adaptive sampling tailored specifically for training LLMs. The method not only achieves faster convergence per iteration but also demonstrates improved scalability compared to traditional SGD techniques. It leverages sample complexity analysis to adaptively choose the sample size, employs a stochastic conjugate subgradient approach to determine search directions and utilizing an AdamW-like algorithm to adaptively adjust step sizes. This approach preserves the key advantages of first-order methods while effectively addressing the nonconvexity and non-smoothness inherent in LLMs training. Additionally, we provide a detailed analysis of the advantage of the algorithm. Experimental results show that the proposed method not only maintains, but in many cases surpasses, the scalability of traditional SGD techniques, significantly enhancing both the speed and accuracy of the optimization process.

cs.LG↗

arXiv2025

Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference

Learning reward models from human preference datasets and subsequently optimizing language models via reinforcement learning has emerged as a fundamental paradigm for aligning LLMs with human preferences. The performance of the reward model plays a crucial role in the effectiveness of alignment. Previous reward models operate at a coarse-grained level, requiring the generation of a complete response to obtain a reward value. The sparse reward may present challenges for downstream reinforcement learning. While recent efforts have attempted to learn token-level reward models, the lack of explicit semantic information makes it difficult to model the credit of every individual token. In this paper, we propose assigning scores to every sentence, introducing an intermediate-grained reward model. By segmenting the complete response into sentences and applying differential operations to reward output at the start and end positions of each sentence, we can effectively model the rewards of sentences. Moreover, a novel attention mechanism is introduced to aggregate the scores of all sentences into a response-level score, which allows it to be trained using the Bradley-Terry model. On common benchmarks, our method outperforms the response-level reward model by 2.7% on RewardBench (for reward modeling evaluation) and surpasses all baselines on AlpacaEval (for alignment evaluation).

cs.CL↗

arXiv2025

The $M_{\rm BH}-M_\star$ Relation of the hyperluminous Dust-obscured Quasars up to $z \sim 4$

Hot dust-obscured galaxies (Hot DOGs) are a rare population of hyperluminous dust-obscured quasars discovered by the Wide-field Infrared Survey Explorer (WISE) all-sky survey. The heavy circumnuclear dust obscuration allows only a small amount of scattered light from the obscured quasar to escape, enabling the decomposition of the stellar component from the total flux. The presence of scattered light enables the redshift of the source and the properties of the black hole to be obtained from SDSS and SDSS-related literature. From WISE and SDSS data, we select 11 hyperluminous Hot DOGs at $z=1.5-3.7$ with bolometric luminosities $L_{\rm bol} \gtrsim 10^{47}\,\mathrm{erg \ s^{-1}}$. We investigate the $M_{\rm BH}-M_\star$ relation in these sources using Bayesian spectral energy distribution (SED) fitting or with extra constraints from \textit{Hubble Space Telescope} (HST) image decomposition. Stellar masses are successfully derived for eight Hot DOGs. We find high Eddington ratios $λ_{\rm Edd}$ in these Hot DOGs, with the median value of 1.05 and the maximum value close to 3. The super-Eddington accretion may be associated with the overdense environments of Hot DOGs. We find no significant differences in the $M_{\rm BH}/M_\star$ of these Hot DOGs compared to the local relation, suggesting that these dust-obscured quasars are the progenitors of massive early-type galaxies. We speculate that the subsequent evolution of Hot DOGs may be significantly influenced by AGN feedback and remain on the local relation.

astro-ph.GA↗

arXiv2025

An Adaptive Sampling-based Progressive Hedging Algorithm for Stochastic Programming

The progressive hedging algorithm (PHA) is a cornerstone among algorithms for large-scale stochastic programming problems. However, its traditional implementation is hindered by some limitations, including the requirement to solve all scenario subproblems in each iteration, reliance on an explicit probability distribution, and a convergence process that is highly sensitive to the choice of certain penalty parameters. This paper introduces a sampling-based PHA which aims to overcome these limitations. Our approach employs a dynamic selection process for the number of scenario subproblems solved per iteration. It incorporates adaptive sequential sampling for determining sample sizes, a stochastic conjugate subgradient method for direction finding, and a line-search technique to update the dual variables. Experimental results demonstrate that this novel algorithm not only addresses the bottlenecks of the conventional PHA but also potentially surpasses its scalability, representing a substantial improvement in the field of stochastic programming.

math.OC↗

arXiv2025

A Comprehensive Survey on Imbalanced Data Learning

With the expansion of data availability, machine learning (ML) has achieved remarkable breakthroughs in both academia and industry. However, imbalanced data distributions are prevalent in various types of raw data and severely hinder the performance of ML by biasing the decision-making processes. To deepen the understanding of imbalanced data and facilitate the related research and applications, this survey systematically analyzes various real-world data formats and concludes existing researches for different data formats into four distinct categories: data re-balancing, feature representation, training strategy, and ensemble learning. This structured analysis helps researchers comprehensively understand the pervasive nature of imbalance across diverse data formats, thereby paving a clearer path toward achieving specific research goals. We provide an overview of relevant open-source libraries, spotlight current challenges, and offer novel insights aimed at fostering future advancements in this critical area of study.

cs.LG↗

arXiv2025

SSMLoRA: Enhancing Low-Rank Adaptation with State Space Model

Fine-tuning is a key approach for adapting language models to specific downstream tasks, but updating all model parameters becomes impractical as model sizes increase. Parameter-Efficient Fine-Tuning (PEFT) methods, such as Low-Rank Adaptation (LoRA), address this challenge by introducing additional adaptation parameters into pre-trained weight matrices. However, LoRA's performance varies across different insertion points within the model, highlighting potential parameter inefficiency due to unnecessary insertions. To this end, we propose SSMLoRA (State Space Model Low-Rank Adaptation), an extension of LoRA that incorporates a State Space Model (SSM) to interconnect low-rank matrices. SSMLoRA ensures that performance is maintained even with sparser insertions. SSMLoRA allows the model to not only map inputs to a low-rank space for better feature extraction but also leverage the computations from the previous low-rank space. Our method achieves comparable performance to LoRA on the General Language Understanding Evaluation (GLUE) benchmark while using only half the parameters. Additionally, due to its structure, SSMLoRA shows promise in handling tasks with longer input sequences. .You can find our code here:https://github.com/yuhkalhic/SSMLoRA.

cs.CL↗

arXiv2025

Myocardial T1 mapping at 5T using multi-inversion recovery real-time spoiled GRE

Objective: To develop an accurate myocardial T1 mapping technique at 5T using Look-Locker-based multiple inversion-recovery with the real-time spoiled gradient echo (GRE) acquisition. Approach: The proposed T1 mapping technique (mIR-rt) samples the recovery of inverted magnetization using the real-time GRE and the images captured during diastole are selected for T1 fitting. Multiple-inversion recoveries are employed to increase the sample size for accurate fitting. The T1 mapping method was validated using Bloch simulation, phantom studies, and in 16 healthy volunteers at 5T. Main results: In both simulation and phantom studies, the T1 values measured by mIR-rt closely approximate the reference T1 values, with errors less than 3%, while the conventional MOLLI sequence underestimates T1 values. The myocardial T1 values at 5T are 1553 +/- 52 ms, 1531 +/- 53 ms, and 1526 +/- 60 ms (mean +/- standard deviation) at the apex, middle, and base, respectively. The T1 values measured by MOLLI (1350 +/- 48 ms, 1349 +/- 47 ms, and 1354 +/- 45 ms at the apex, middle, and base) were significantly lower than those of mIR-rt with p<0.001 for all three layers. The mIR-rt sequence method used in our study provides high reproducibility, particularly in the middle slices, supporting its practical relevance for myocardial T1 mapping. Significance: The proposed method is feasible for myocardial T1 mapping at 5T and provides better accuracy than the conventional MOLLI sequence.

physics.med-ph↗

arXiv2024

A new approach to constraining properties of AGN host galaxies by combining image and SED decomposition: testing upon the MBH-M* relation

The outshining light from active galactic nuclei (AGNs) poses significant challenges in studying the properties of AGN host galaxies. To address this issue, we propose a novel approach which combines image decomposition and spectral energy distribution (SED) decomposition to constrain properties of AGN host galaxies. Image decomposition allows us to disentangle optical flux into AGN and stellar components, thereby providing additional constraints on the SED models to derive more refined stellar mass. To test the viability of this approach, we obtained a sample of 24 X-ray selected type-I AGNs with redshifts ranging from 0.73 to 2.47. We estimated the stellar masses for our sample and found that our results are generally consistent with earlier estimates based on different methods. Through examining the posterior distribution of stellar masses, we find that our method could derive better constrained results compared to previous SED decomposition methods. With the derived stellar masses, we further studied the MBH-M* relation of our sample, finding a higher intrinsic scatter in the correlation for our entire sample compared to the local quiescent correlation, which could be caused by a few black hole monsters in our sample. We propose that based on our method, future works could extend to larger samples of high-redshift AGN host galaxies, thereby enhancing our understanding of their properties.

astro-ph.GA↗