arXiv subjects

Yong Zhang

Publications and source records attributed to Yong Zhang.

At least 145 records · Page 8Linked to original sources

arXiv2025

Cross-attention Secretly Performs Orthogonal Alignment in Recommendation Models

Cross-domain sequential recommendation (CDSR) aims to align heterogeneous user behavior sequences collected from different domains. While cross-attention is widely used to enhance alignment and improve recommendation performance, its underlying mechanism is not fully understood. Most researchers interpret cross-attention as residual alignment, where the output is generated by removing redundant and preserving non-redundant information from the query input by referencing another domain data which is input key and value. Beyond the prevailing view, we introduce Orthogonal Alignment, a phenomenon in which cross-attention discovers novel information that is not present in the query input, and further argue that those two contrasting alignment mechanisms can co-exist in recommendation models We find that when the query input and output of cross-attention are orthogonal, model performance improves over 300 experiments. Notably, Orthogonal Alignment emerges naturally, without any explicit orthogonality constraints. Our key insight is that Orthogonal Alignment emerges naturally because it improves scaling law. We show that baselines additionally incorporating cross-attention module outperform parameter-matched baselines, achieving a superior accuracy-per-model parameter. We hope these findings offer new directions for parameter-efficient scaling in multi-modal research.

cs.LG↗

arXiv2025

PDRs4All XIX. The 6 to 9 $μ$m region as a probe of PAH charge and size in the Orion Bar

Infrared emission from polycyclic aromatic hydrocarbons (PAHs) play a major role in determining the charge balance of their host environments that include photo-dissociation regions (PDRs) in galaxies, planetary nebulae, and rims of molecular clouds. We aim to investigate the distribution and sizes of charged PAHs across the key zones of the Orion Bar PDR. We employ JWST MIRI-MRS observations of the Orion Bar from the Early Release Science program ''PDRs4All'' and synthetic images in the JWST MIRI filters. We investigate the spatial morphology of the AIBs at 6.2, 7.7, 8.6, and 11.0 $μ$m that commonly trace PAH cations, and the neutral PAH-tracing 11.2 $μ$m AIB, their (relative) correlations, and the relationship with existing empirical prescriptions for AIBs. The 6.2. 7.7, 8.6, 11.0, and 11.2 $μ$m AIBs are similar in spatial morphology, on larger scales. Analyzing three-feature intensity correlations, two distinct groups emerge: the 8.6 and 11.0 $μ$m vs. the 6.2 and 7.7 $μ$m AIBs. We attribute these correlations to PAH size. The 6.2 and 7.7 $μ$m AIBs trace cationic, medium-sized PAHs. Quantum chemical calculations reveal that the 8.6 $μ$m AIB is carried by large, compact, cationic PAHs, and the 11.0 $μ$m AIB's correlation to it implies, so is this band. The 6.2/8.6 and 7.7/8.6 PAH band ratios thus probe PAH size. We conclude that the 6.2/11.2 AIB ratio is the most reliable proxy for charged PAHs, within the cohort. We outline JWST MIRI imaging prescriptions that serve as effective tracers of the PAH ionization fraction as traced. This study showcases the efficacy of the 6-9 $μ$m AIBs to probe the charge state and size distribution of the emitting PAHs, offering insights into the physical conditions of their host environments. JWST MIRI photometry offers a viable alternative to IFU spectroscopy for characterizing this emission in extended objects.

astro-ph.GA↗

arXiv2025

Do LLMs Align with My Task? Evaluating Text-to-SQL via Dataset Alignment

Supervised Fine-Tuning (SFT) is an effective method for adapting Large Language Models (LLMs) on downstream tasks. However, variability in training data can hinder a model's ability to generalize across domains. This paper studies the problem of dataset alignment for Natural Language to SQL (NL2SQL or text to SQL), examining how well SFT training data matches the structural characteristics of target queries and how this alignment impacts model performance. We hypothesize that alignment can be accurately estimated by comparing the distributions of structural SQL features across the training set, target data, and the model's predictions prior to SFT. Through comprehensive experiments on three large cross-domain NL2SQL benchmarks and multiple model families, we show that structural alignment is a strong predictor of fine-tuning success. When alignment is high, SFT yields substantial gains in accuracy and SQL generation quality; when alignment is low, improvements are marginal or absent. These findings highlight the importance of alignment-aware data selection for effective fine-tuning and generalization in NL2SQL tasks.

cs.CL↗

arXiv2025

ElasticMoE: An Efficient Auto Scaling Method for Mixture-of-Experts Models

Mixture-of-Experts (MoE) models promise efficient scaling of large language models (LLMs) by activating only a small subset of experts per token, but their parallelized inference pipelines make elastic serving challenging. Existing strategies fall short: horizontal scaling provisions entire replicas of the current configuration, often tens to hundreds of accelerators, leading to coarse granularity, long provisioning delays, and costly overprovisioning. Vertical scaling offers finer adjustments but typically requires instance restarts, incurring downtime. These limitations make current approaches ill-suited for the bursty, short-lived traffic patterns common in cloud deployments. We present ElasticMoE, an elastic scaling framework for MoE LLMs that achieves fine-grained, low-latency, and zero-downtime scaling. ElasticMoE decouples inference execution from memory operations, enabling scaling steps to proceed concurrently with serving. An HBM Management Module (HMM) reuses weights and KV caches via zero-copy remapping, while high-bandwidth peer-to-peer transfers bring newly added accelerators online without interrupting service. A virtual memory based expert redistribution mechanism migrates MoE experts without costly buffer reallocations, reducing peak memory usage during expert parallelism reconfiguration. Our evaluation on Ascend NPUs with three popular MoE LLMs shows that ElasticMoE achieves up to 9x lower scale-up latency, up to 2x better throughput during scaling, and significantly improves SLO attainment compared to baselines. By enabling fine-grained, concurrent scaling with minimal disruption, ElasticMoE advances the practicality of deploying massive MoE LLMs in dynamic cloud environments.

cs.DC↗

arXiv2025

Spatial Reasoning with Vision-Language Models in Ego-Centric Multi-View Scenes

Understanding 3D spatial relationships remains a major limitation of current Vision-Language Models (VLMs). Prior work has addressed this issue by creating spatial question-answering (QA) datasets based on single images or indoor videos. However, real-world embodied AI agents such as robots and self-driving cars typically rely on ego-centric, multi-view observations. To this end, we introduce Ego3D-Bench, a new benchmark designed to evaluate the spatial reasoning abilities of VLMs using ego-centric, multi-view outdoor data. Ego3D-Bench comprises over 8,600 QA pairs, created with significant involvement from human annotators to ensure quality and diversity. We benchmark 16 SOTA VLMs, including GPT-4o, Gemini1.5-Pro, InternVL3, and Qwen2.5-VL. Our results reveal a notable performance gap between human level scores and VLM performance, highlighting that current VLMs still fall short of human level spatial understanding. To bridge this gap, we propose Ego3D-VLM, a post-training framework that enhances 3D spatial reasoning of VLMs. Ego3D-VLM generates cognitive map based on estimated global 3D coordinates, resulting in 12% average improvement on multi-choice QA and 56% average improvement on absolute distance estimation. Ego3D-VLM is modular and can be integrated with any existing VLM. Together, Ego3D-Bench and Ego3D-VLM offer valuable tools for advancing toward human level spatial understanding in real-world, multi-view environments.

cs.CV↗

arXiv2025

FAST search for circumstellar atomic hydrogen. IV. bubbles associated with planetary nebulae

Investigating the bubbles generated by the interaction between asymptotic giant branch stellar outflows and the interstellar medium (ISM) is pivotal for elucidating the mechanism by which evolved low- to intermediate-mass stars enrich the ISM with heavy elements. Using archival datasets from the Galactic Plane Pulsar Snapshot survey and the Galactic Arecibo L-Band Array \ion{H}{1} survey, we have identified 14 bubbles within interstellar atomic hydrogen (\ion{H}{1}) maps, each showing evidence of potential association with planetary nebulae (PNe).We pursue two primary objectives centered on the identified ISM bubbles and their association with PNe. First, leveraging the calibrated distance measurements of PNe from Gaia Data Release 3, we utilize these ISM bubbles as observational tracers to investigate and constrain the Galactic rotation curve. Second, we note that distance determinations for some PNe remain unreliable, partly because their central stars are obscured by extended nebular envelopes or are misidentified. Therefore, we develop a novel methodological framework to derive kinematic distances for PNe by leveraging the velocities of their associated ISM bubbles and constraints from the Galactic rotation curve.

astro-ph.SR↗

arXiv2025

Loop-Level Double Copy Relations from Forward Limits

We study double copy relations for loop integrands in gauge theories and gravity based on their constructions from single cuts, which are in turn obtained from forward limits of lower-loop cases. While such a construction from forward limits has been realized for loop integrands in gauge theories, we demonstrate its extension to gravity by reconstructing one-loop gravity integrands from forward limits of trees. Under mild symmetry assumptions on tree-level kinematic numerators (and their forward limits), our method directly leads to double copy relations for one-loop integrands: these include the field-theoretic Kawai-Lewellen-Tye (KLT) relations, whose kernel is the inverse of a matrix with rank $(n{-}1)!$ formed by those in bi-adjoint $ϕ^3$ theory, and the Bern-Carrasco-Johansson (BCJ) double copy relations with crossing-symmetric kinematic numerators (we provide local and crossing-symmetric Yang-Mills BCJ numerators for $n=3,4,5$ explicitly). By exploiting the "universal expansion" for one-loop integrands in generic gauge theories, we also obtain an analogous expansion for gravity (including supergravity theories).

hep-th↗

arXiv2025

Accelerate Creation of Product Claims Using Generative AI

The benefit claims of a product is a critical driver of consumers' purchase behavior. Creating product claims is an intense task that requires substantial time and funding. We have developed the $\textbf{Claim Advisor}$ web application to accelerate claim creations using in-context learning and fine-tuning of large language models (LLM). $\textbf{Claim Advisor}$ was designed to disrupt the speed and economics of claim search, generation, optimization, and simulation. It has three functions: (1) semantically searching and identifying existing claims and/or visuals that resonate with the voice of consumers; (2) generating and/or optimizing claims based on a product description and a consumer profile; and (3) ranking generated and/or manually created claims using simulations via synthetic consumers. Applications in a consumer packaged goods (CPG) company have shown very promising results. We believe that this capability is broadly useful and applicable across product categories and industries. We share our learning to encourage the research and application of generative AI in different industries.

cs.AI↗

arXiv2025

A Survey of Large Language Models for Data Challenges in Graphs

Graphs are a widely used paradigm for representing non-Euclidean data, with applications ranging from social network analysis to biomolecular prediction. While graph learning has achieved remarkable progress, real-world graph data presents a number of challenges that significantly hinder the learning process. In this survey, we focus on four fundamental data-centric challenges: (1) Incompleteness, real-world graphs have missing nodes, edges, or attributes; (2) Imbalance, the distribution of the labels of nodes or edges and their structures for real-world graphs are highly skewed; (3) Cross-domain Heterogeneity, graphs from different domains exhibit incompatible feature spaces or structural patterns; and (4) Dynamic Instability, graphs evolve over time in unpredictable ways. Recently, Large Language Models (LLMs) offer the potential to tackle these challenges by leveraging rich semantic reasoning and external knowledge. This survey focuses on how LLMs can address four fundamental data-centric challenges in graph-structured data, thereby improving the effectiveness of graph learning. For each challenge, we review both traditional solutions and modern LLM-driven approaches, highlighting how LLMs contribute unique advantages. Finally, we discuss open research questions and promising future directions in this emerging interdisciplinary field. To support further exploration, we have curated a repository of recent advances on graph learning challenges: https://github.com/limengran98/Awesome-Literature-Graph-Learning-Challenges.

cs.LG↗

arXiv2025

Dynamic Adaptive Parsing of Temporal and Cross-Variable Patterns for Network State Classification

Effective network state classification is a primary task for ensuring network security and optimizing performance. Existing deep learning models have shown considerable progress in this area. Some methods excel at analyzing the complex temporal periodicities found in traffic data, while graph-based approaches are adept at modeling the dynamic dependencies between different variables. However, a key trade-off remains, as these methods struggle to capture both characteristics simultaneously. Models focused on temporal patterns often overlook crucial variable dependencies, whereas those centered on dependencies may fail to capture fine-grained temporal details. To address this trade-off, we introduce DAPNet, a framework based on a Mixture-of-Experts architecture. DAPNet integrates three specialized networks for periodic analysis, dynamic cross-variable correlation modeling, and hybrid temporal feature extraction. A learnable gating network dynamically assigns weights to experts based on the input sample and computes a weighted fusion of their outputs. Furthermore, a hybrid regularization loss function ensures stable training and addresses the common issue of class imbalance. Extensive experiments on two large-scale network intrusion detection datasets (CICIDS2017/2018) validate DAPNet's higher accuracy for its target application. The generalizability of the architectural design is evaluated across ten public UEA benchmark datasets, positioning DAPNet as a specialized framework for network state classification.

cs.LG↗

arXiv2025

RAM++: Robust Representation Learning via Adaptive Mask for All-in-One Image Restoration

This work presents Robust Representation Learning via Adaptive Mask (RAM++), a two-stage framework for all-in-one image restoration. RAM++ integrates high-level semantic understanding with low-level texture generation to achieve content-oriented robust restoration. It addresses the limitations of existing degradation-oriented methods in extreme scenarios (e.g., degradations strongly coupled with image structures). RAM++ also mitigates common challenges such as unbalanced performance across tasks, overfitting to seen degradations, and weak generalization to unseen ones through three key designs: 1) Adaptive Semantic-Aware Mask (AdaSAM): a pretraining strategy that applies pixel-level masks to semantically rich and textured regions. This design enables the network to learn both generative priors and image content priors from various degradations. 2) Mask Attribute Conductance (MAC): a selective fine-tuning strategy that adjusts the layers with higher contributions to bridge the integrity gap between masked pretraining and full-image fine-tuning while retaining learned priors. 3) Robust Feature Regularization (RFR): a strategy that leverages DINOv2's semantically consistent and degradation-invariant representations, together with efficient feature fusion, to achieve faithful and semantically coherent restoration. With these designs, RAM++ achieves robust, well-balanced, and state-of-the-art performance across seen, unseen, extreme, and mixed degradations. Our code and model will be released at https://github.com/DragonisCV/RAM

cs.CV↗

arXiv2025

LV-CadeNet: A Long-View Feature Convolution-Attention Fusion Encoder-Decoder Network for EEG/MEG Spike Analysis

The analysis of interictal epileptiform discharges (IEDs) in magnetoencephalography (MEG) or electroencephalogram (EEG) recordings represents a critical component in the diagnosis of epilepsy. However, manual analysis of these IEDs, which appear as epileptic spikes, from the large amount of MEG/EEG data is labor intensive and requires high expertise. Although automated methods have been developed to address this challenge, current approaches fail to fully emulate clinical experts' diagnostic intelligence in two key aspects: (1) their analysis on the input signals is limited to short temporal windows matching individual spike durations, missing the extended contextual patterns clinicians use to assess significance; and (2) they fail to adequately capture the dipole patterns with simultaneous positive-negative potential distributions across adjacent sensors that serve as clinicians' key diagnostic criterion for IED identification. To bridge this artificial-human intelligence gap, we propose a novel deep learning framework LV-CadeNet that integrates two key innovations: (1) a Long-View morphological feature representation that mimics expert clinicians' comprehensive assessment of both local spike characteristics and long-view contextual information, and (2) a hierarchical Encoder-Decoder NETwork that employs Convolution-Attention blocks for multi-scale spatiotemporal feature learning with progressive abstraction. Extensive evaluations confirm the superior performance of LV-CadeNet, which outperforms six state-of-the-art methods in EEG spike classification on TUEV, the largest public EEG spike dataset. Additionally, LV-CadeNet attains a significant improvement of 13.58% in balanced accuracy over the leading baseline for MEG spike detection on a clinical MEG dataset from Sanbo Brain Hospital, Capital Medical University.

cs.CV↗

arXiv2025

Cocoon-to-Butterfly Transformation in Protoplanetary Nebula: Detection of a Spherical Halo and a Barrel-shaped Torus in IRAS 06530$-$0213

Protoplanetary nebulae (PPNs) represent a critical evolutionary bridging stage between the asymptotic giant branch (AGB) and planetary nebula (PN) stages. Their dynamical structures provide key insights into late stellar evolution. Here, we report CO and $^{13}$CO imaging observations of the carbon-rich PPN IRAS 06530$-$0213, a source exhibiting the unidentified 21 micron emission band, conducted with the Northern Extended Millimeter Array (NOEMA). The CO maps reveal a spherical halo (diameter $\sim10''$) surrounding a central barrel-shaped torus, where the torus displays an inner diameter of $\sim1.5''$ and an outer diameter of $\sim2''$. Through three-dimensional morpho-kinematic modeling with the SHAPE software, we determine that IRAS 06530$-$0213 experienced its final thermal pulse during the AGB phase $\sim6500$ years ago, transitioning into the PPN phase $\sim4500$ years ago. Our analysis indicates that the blue-shifted CO emission, a feature also detected in several other PPNs and PNs previously misattributed to the interstellar clouds, actually originates from the obscuration of the central nebula by the remnant AGB halo. These findings are expected to deepen our understanding of the dynamical structures of PPNs, as well as their pivotal transitional role in the late-stage evolution of low-to-intermediate mass stars.

astro-ph.SR↗

arXiv2025

ExpertWeave: Efficiently Serving Expert-Specialized Fine-Tuned Adapters at Scale

Expert-Specialized Fine-Tuning (ESFT) adapts Mixture-of-Experts (MoE) large language models to enhance their task-specific performance by selectively tuning the top-activated experts for the task. Serving these fine-tuned models at scale is challenging: deploying merged models in isolation is prohibitively resource-hungry, while existing multi-adapter serving systems with LoRA-style additive updates are incompatible with ESFT's expert-oriented paradigm. We present ExpertWeave, a system that serves multiple ESFT adapters concurrently over a single shared MoE base model, drastically reducing the memory footprint and improving resource utilization. To seamlessly integrate into existing inference pipelines for MoE models with non-intrusive modifications and minimal latency overhead, ExpertWeave introduces a virtual-memory-assisted expert weight manager that co-locates base-model and adapter experts without incurring memory overhead from fragmentation, and a fused kernel for batched rerouting to enable lightweight redirection of tokens to the appropriate experts at runtime. Our evaluations show that ExpertWeave can simultaneously serve multiple adapters of a 16B MoE model on a single accelerator where the baseline runs out of memory, or provides up to 94x more KV cache capacity and achieves up to 18% higher throughput while using comparable resources, all without compromising model accuracy. ExpertWeave maintains low overhead even when scaling to 20 adapters, with a 4-11% latency increase compared with serving the base model alone. Source code will be released soon.

cs.DC↗

arXiv2025

A Lightweight Group Multiscale Bidirectional Interactive Network for Real-Time Steel Surface Defect Detection

Real-time surface defect detection is critical for maintaining product quality and production efficiency in the steel manufacturing industry. Despite promising accuracy, existing deep learning methods often suffer from high computational complexity and slow inference speeds, which limit their deployment in resource-constrained industrial environments. Recent lightweight approaches adopt multibranch architectures based on depthwise separable convolution (DSConv) to capture multiscale contextual information. However, these methods often suffer from increased computational overhead and lack effective cross-scale feature interaction, limiting their ability to fully leverage multiscale representations. To address these challenges, we propose GMBINet, a lightweight framework that enhances multiscale feature extraction and interaction through novel Group Multiscale Bidirectional Interactive (GMBI) modules. The GMBI adopts a group-wise strategy for multiscale feature extraction, ensuring scale-agnostic computational complexity. It further integrates a Bidirectional Progressive Feature Interactor (BPFI) and a parameter-free Element-Wise Multiplication-Summation (EWMS) operation to enhance cross-scale interaction without introducing additional computational overhead. Experiments on SD-Saliency-900 and NRSD-MN datasets demonstrate that GMBINet delivers competitive accuracy with real-time speeds of 1048 FPS on GPU and 16.53 FPS on CPU at 512 resolution, using only 0.19 M parameters. Additional evaluations on the NEU-CLS defect classification dataset further confirm the strong generalization ability of our method, demonstrating its potential for broader industrial vision applications beyond surface defect detection. The dataset and code are publicly available at: https://github.com/zhangyongcode/GMBINet.

cs.CV↗

arXiv2025

Fair and efficient contribution valuation for vertical federated learning

Federated learning is an emerging technology for training machine learning models across decentralized data sources without sharing data. Vertical federated learning, also known as feature-based federated learning, applies to scenarios where data sources have the same sample IDs but different feature sets. To ensure fairness among data owners, it is critical to objectively assess the contributions from different data sources and compensate the corresponding data owners accordingly. The Shapley value is a provably fair contribution valuation metric originating from cooperative game theory. However, its straight-forward computation requires extensively retraining a model on each potential combination of data sources, leading to prohibitively high communication and computation overheads due to multiple rounds of federated learning. To tackle this challenge, we propose a contribution valuation metric called vertical federated Shapley value (VerFedSV) based on the classic Shapley value. We show that VerFedSV not only satisfies many desirable properties of fairness but is also efficient to compute. Moreover, VerFedSV can be adapted to both synchronous and asynchronous vertical federated learning algorithms. Both theoretical analysis and extensive experimental results demonstrate the fairness, efficiency, adaptability, and effectiveness of VerFedSV.

cs.LG↗

arXiv2025

The Missing Giant: Do FAST Spectroscopic Observations Reveal a Scarcity of Large Polycyclic Aromatic Hydrocarbons in Astronomical Environments?

The search for large polycyclic aromatic hydrocarbons (PAHs) with over 100 carbon atoms is crucial to resolving the origin of unidentified infrared emission (UIE) bands. These bands are commonly observed in nebulae and the interstellar medium, yet their spectroscopic assignment has remained unknown for decades. Using the Five-hundred-meter Aperture Spherical Radio Telescope (FAST), the world's most sensitive instrument operating in the decimeter-wavelength range, we conducted a search for rotational transitions of large, quasi-symmetric PAHs. Our sample included two prototypical UIE sources, NGC 7027 and TMC-1, along with a non-UIE source, IRC+10216, for comparison. A matched filter technique was employed to isolate comb-like spectral features from quasi-symmetric PAHs containing 138 to 194 carbon atoms in the FAST spectra. This method significantly enhanced detection sensitivity to these astrophysically critical molecular signatures. Although no such features were detected, we derived upper limits on the abundance of large PAHs based on simplifying assumptions. These upper limits are lower than the values predicted by theoretical models, which might tentatively suggest that large PAHs may not be the primary carriers of UIE bands. However, this conclusion should be treated as tentative, given that it rests on simplistic assumptions which have not been empirically validated.

astro-ph.GA↗

arXiv2025

From Near-Integrable to Far-from-Integrable: A Unified Picture of Thermalization and Heat Transport

Whether and how a system approaches equilibrium is central in nonequilibrium statistical physics, crucial to understanding thermalization and transport. Bogoliubov's three-stage (initial, kinetic, and hydrodynamic) evolution hypothesis offers a qualitative framework, but quantitative progress has focused on near-integrable systems like dilute gases. In this work, we investigate the relaxation dynamics of a one-dimensional diatomic hard-point (DHP) gas, presenting a phase diagram that characterizes relaxation behavior across the full parameter space, from near-integrable to far-from-integrable regimes. We analyze thermalization (local energy relaxation in nonequilibrium states) and identify three universal dynamical regimes: (i) In the near-integrable regime, kinetic processes dominate, local energy relaxation decays exponentially, and the thermalization time $τ$ scales as $τ\propto δ^{-2}$. (ii) In the far-from-integrable regime, hydrodynamic effects dominate, energy relaxation decays power-law, and thermalization time scales linearly with system size $N$. (iii) In the intermediate regime, the Bogoliubov phase emerges, characterized by the transition from kinetic to hydrodynamic relaxation. The phase diagram also shows that hydrodynamic behavior can emerge in small systems when sufficiently far from the integrable regime, challenging the view that such effects occur only in large systems. In the thermodynamic limit, the system's relaxation depends on the order in which the limits ($N \to \infty$ or $δ\to 0$) are taken. We then analyze heat transport (decay of heat-current fluctuations in equilibrium), demonstrating its consistency with thermalization, leading to a unified theoretical description of thermalization and transport. Our approach provides a pathway for studying relaxation dynamics in many-body systems, including quantum systems.

cond-mat.stat-mech↗